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DEFENSE  LOGISTICS 

Improvements  Needed  to  Enhance  Oversight  of 
Estimated  Long-term  Costs  for  Operating  and 
Supporting  Major  Weapon  Systems 


Why  GAO  Did  This  Study 

With  the  nation  facing  fiscal  challenges 
and  the  potential  for  tighter  defense 
budgets,  Congress  and  the 
Department  of  Defense  (DOD)  have 
placed  more  attention  on  controlling 
the  billions  of  dollars  spent  annually  on 
weapon  system  operating  and  support 
(O&S)  costs.  These  costs  include, 
costs  for  repair  parts,  maintenance, 
and  personnel,  and  account  for  about 
70  percent  of  the  total  costs  of  a 
weapon  system  over  its  life  cycle.  The 
selected  acquisition  report  (SAR)  is 
DOD’s  key  recurring  status  report  on 
the  cost,  schedule,  and  performance  of 
major  defense  acquisition  programs 
and  is  intended  to  provide  authoritative 
information  for  congressional  oversight 
of  these  programs.  Oversight  of  O&S 
costs  is  important  because  many  of  the 
key  decisions  affecting  these  life-cycle 
costs  are  made  during  the  acquisition 
process.  GAO  reviewed  weapon 
system  O&S  cost  estimates  that  DOD 
submits  in  the  SAR.  Specifically,  GAO 
determined  the  extent  to  which  the 
SARs  provide  consistent  and  reliable 
O&S  cost  estimate  information  that 
enables  effective  oversight  of  these 
weapon  system  costs.  To  conduct  its 
review,  GAO  analyzed  SAR  data  for  84 
major  systems  that  submitted  O&S 
cost  estimates  in  the  2010  SAR  and 
selected  a  nonprobability  sample  of  15 
systems  for  more  in-depth  review. 

What  GAO  Recommends 

To  enhance  visibility  of  weapon  system 
O&S  costs  during  acquisition,  GAO 
recommends  that  DOD  improve  its 
guidance  to  program  offices  on  cost 
reporting  and  also  improve  its  process 
for  reviewing  these  costs  prior  to  final 
submission  of  the  SAR  to  Congress. 
DOD  concurred  with  GAO’s 
recommendations. 

View  GAO-12-340.  For  more  information, 
contact  Cary  B.  Russell  at  (404)  679-1808  or 
russellc@gao.gov. 


What  GAO  Found 

DOD’s  reports  to  Congress  on  estimated  weapon  system  O&S  costs  are  often 
inconsistent  and  sometimes  unreliable,  limiting  visibility  needed  for  effective 
oversight  of  these  costs.  The  SAR  statute  requires  that  life-cycle  cost  reporting 
for  major  weapon  systems  be  uniform,  to  the  extent  practicable,  across  the 
department,  but  GAO  found  a  number  of  inconsistent  practices  in  how  program 
offices  were  reporting  life-cycle  O&S  cost  estimates  in  the  SAR.  Program  offices 
were  inconsistent  in  (1)  the  explanatory  information  they  included  with  the  cost 
estimates;  (2)  the  source  of  the  cost  estimate  they  cited  as  the  basis  for  the 
reported  costs;  (3)  the  unit  of  measure  they  used  to  portray  average  costs; 

(4)  the  frequency  with  which  they  updated  reported  costs;  and  (5)  the  reporting  of 
costs  for  an  antecedent  system  being  replaced  by  the  new  weapon  system.  For 
example,  35  (42  percent)  of  the  84  programs  that  reported  O&S  costs  in  the  2010 
SAR  did  not  cite  a  source  of  these  data,  contrary  to  DOD’s  guidance,  and  57  (68 
percent)  of  the  programs  did  not  report  O&S  costs  for  an  antecedent  system. 
Also,  O&S  cost  submissions  in  the  SAR  did  not  always  incorporate  best  practices 
for  presenting  cost  estimates,  such  as  tracking  cost  changes  over  time  and 
identifying  cost  drivers.  In  addition,  1 1  systems  did  not  provide  O&S  cost 
estimates  in  the  2010  SAR. 

Although  SARs  are  intended  to  provide  Congress  with  authoritative  program 
information  on  major  weapon  systems,  7  of  the  15  sample  programs  GAO 
reviewed  submitted  unreliable  O&S  cost  estimate  data  in  the  2007,  2009,  or  2010 
SARs.  For  example,  an  Air  Force  program  underreported  O&S  costs  by  $2.1 
billion  (fiscal  year  2002  dollars),  or  18  percent.  While  some  of  the  program  offices 
did  not  provide  an  explanation  for  the  errors  in  the  submitted  data,  others  cited 
specific  reasons.  For  example,  one  Navy  program  office  underreported  O&S 
costs  in  the  SAR  and  explained  that  it  excluded  certain  costs  that  were  not  under 
its  control,  such  as  externally  funded  spare  parts  and  military  personnel. 

However,  excluding  such  costs  is  contrary  to  the  SAR  statute.  An  Air  Force 
program  reported  current  and  projected  funding  for  the  program  rather  than 
estimated  life-cycle  O&S  costs.  This  practice  also  had  the  effect  of 
underreporting  these  costs. 

DOD’s  reports  to  Congress  on  estimated  weapon  system  O&S  costs  were  often 
inconsistent  and  sometimes  unreliable  due  to  a  lack  of  (1 )  detailed 
implementation  guidance  for  reporting  these  costs  and  (2)  an  effective  process 
for  reviewing  the  O&S  cost  sections  of  the  SAR  before  final  submission  to 
Congress.  DOD’s  guidance  collectively  provides  minimal  instructions  for  O&S 
cost  reporting.  The  guidance  also  does  not  incorporate  some  of  the  best 
practices  GAO  has  identified  for  presenting  cost  estimates.  Further,  although  the 
SAR  data  submitted  by  program  offices  are  subject  to  multiple  reviews  within  the 
military  services  and  by  the  Office  of  the  Secretary  of  Defense,  this  review 
process  has  not  provided  assurance  that  O&S  costs  are  reported  consistently 
and  reliably.  In  the  absence  of  improvements  to  the  SAR  guidance  and  to  the 
review  process,  deficiencies  in  reporting  O&S  costs  are  likely  to  continue. 
Improved  reporting  of  O&S  costs  in  the  SAR  could  help  to  place  more  emphasis 
on  assessing,  managing,  and  controlling  long-term  weapon  system  O&S  costs. 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


February  2,  2012 

The  Honorable  J.  Randy  Forbes 
Chairman 

The  Honorable  Madeleine  Bordallo 
Ranking  Member 
Subcommittee  on  Readiness 
Committee  on  Armed  Services 
House  of  Representatives 

With  the  nation  facing  fiscal  challenges  and  the  potential  for  tighter 
defense  budgets,  both  Congress  and  the  Department  of  Defense  (DOD) 
in  recent  years  have  placed  more  attention  on  controlling  the  billions  of 
dollars  spent  annually  on  weapon  system  operating  and  support  (O&S) 
costs.  These  costs  include,  among  other  things,  costs  for  repair  parts, 
maintenance,  and  personnel,  and  historically  have  accounted  for  about 
70  percent  of  a  weapon  system’s  total  costs.1  In  short,  the  acquisition  of  a 
weapon  system  today  involves  a  significant  financial  commitment  to  that 
system  over  its  entire  life  cycle,  a  period  that  may  last  several  decades 
from  the  system’s  development  to  the  time  it  is  removed  from  DOD’s 
inventory.  For  example,  the  program  office  for  the  F-35  Joint  Strike 
Fighter,  the  newest  aircraft  being  acquired  for  the  Air  Force,  Navy,  and 
Marines,  estimated  in  2010  that  life-cycle  O&S  costs  were  about  $1.0 
trillion,  in  addition  to  an  estimated  $379.4  billion  in  total  acquisition  costs.2 

Since  1969,  the  selected  acquisition  report  (SAR)  has  been  the  key 
recurring  summary  status  report  to  Congress  on  the  cost,  schedule,  and 


1  According  to  DOD  officials,  O&S  costs  generally  range  from  60  to  80  percent  of  a 
weapon  system’s  total  costs,  depending  on  the  weapon  system  type.  DOD  previously 
estimated  that  weapon  system  product  support  costs  in  fiscal  year  2008  were  at  least 
$132  billion.  According  to  DOD,  product  support  encompasses  materiel  management, 
distribution,  technical  data  management,  maintenance,  training,  cataloging,  configuration 
management,  engineering  support,  repair  parts  management,  failure  reporting  and 
analysis,  and  reliability  growth.  Under  this  definition,  product  support  does  not  include  all 
the  costs  categorized  as  O&S  costs. 

2  These  costs  are  expressed  in  then-year  dollars.  Estimated  O&S  costs  in  fiscal  year  2002 
dollars  were  about  $420.3  billion. 
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performance  of  DOD’s  major  defense  acquisition  programs.3  (We  also 
use  the  term  major  weapon  systems  to  refer  to  these  programs,  although 
some  programs  involve  other  types  of  defense  systems.)  The  SAR  is 
intended  to  provide  authoritative  information  needed  for  congressional 
oversight  of  these  programs.  Although  much  of  the  data  reported  within 
the  SAR  is  acquisition  related,  in  1985  Congress  amended  the  SAR 
statute  to  require  that  a  full  life-cycle  cost  analysis  also  be  included  in  the 
SAR,4  and  subsequently  specified  that  this  life-cycle  cost  analysis  include 
estimated  O&S  costs.5  This  information  is  important  for  effective  oversight 
because  many  of  the  key  decisions  affecting  a  weapon  system’s  O&S 
costs  over  its  life  cycle  are  made  while  the  system  is  still  in  the  acquisition 
process.  Acquisition-related  decisions  about  the  design,  materials,  and 
technology  for  the  system  affect  the  logistics  support  that  is  eventually 
needed  to  keep  the  system  available  and  ready  after  it  enters  into  DOD’s 
inventory.  Further,  investments  to  improve  reliability,  availability,  and 
maintainability  during  acquisition  can  reduce  the  future  O&S  costs  of 
DOD’s  weapon  systems.  For  example,  higher  reliability  can  be  designed 
into  the  weapon  system  during  its  development  by  reducing  technical 
requirements,  using  highly  reliable  and  proven  components,  or  investing 
more  in  early  testing,  thus  improving  prospects  for  readiness  and 
reducing  O&S  costs  across  the  life  cycle.  As  the  visibility  and 
management  of  O&S  costs  have  become  more  of  a  focus  in  the 
department,  the  Office  of  the  Secretary  of  Defense  (OSD)  has  recognized 
the  SAR  as  a  source  of  O&S  cost  information  for  oversight  of  weapon 
system  acquisitions.  Therefore,  the  O&S  costs  reported  in  the  SARs  are 


3  Major  defense  acquisition  programs  are  those  estimated  by  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology  and  Logistics  to  require  an  eventual  total 
expenditure,  including  all  planned  increments,  of  more  than  $365  million  for  research, 
development,  test,  and  evaluation,  based  on  fiscal  year  2000  constant  dollars 
(approximately  $509  million  in  fiscal  year  2010  dollars),  $2,190  billion  for  procurement, 
based  on  fiscal  year  2000  constant  dollars  (approximately  $3,054  billion  in  fiscal  year 
2010  dollars),  or  are  designated  as  a  major  defense  acquisition  program  by  the  Milestone 
Decision  Authority.  10  U.S.C.  §  2430;  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  Directive-Type  Memorandum  (DTM)  09-027-lmplementation  of 
the  Weapon  Systems  Acquisition  Reform  Act  of  2009,  attachment  1 ,  para.  1 3  (Dec.  4, 
2009). 

4  Department  of  Defense  Authorization  Act,  1986,  Pub.  L.  No.  99-145,  §  1201  (1985). 

5  See  National  Defense  Authorization  Act  for  Fiscal  Year  1991,  Pub.  L.  No.  101-510,  § 
1407(b)  (1990);  Federal  Acquisition  Streamlining  Act  of  1994,  Pub.  L.  No.  103-355,  § 
3002(c)  (1994). 
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not  only  reported  to  Congress,  but  they  are  also  included  in  internal  OSD 
management  reports  as  well. 

In  response  to  the  Weapon  Systems  Acquisition  Reform  Act  of  2009, 6  we 
previously  reviewed  the  growth  in  O&S  costs  of  major  weapon  systems 
and  reported  that  DOD  lacked  key  information  needed  to  effectively 
manage  and  reduce  O&S  costs — including  life-cycle  O&S  cost  estimates 
and  complete  historical  data  on  actual  O&S  costs.7  DOD  generally 
concurred  with  the  recommendations  in  that  report  and  stated  that  it  was 
taking  corrective  actions.  More  recently,  the  National  Defense 
Authorization  Act  for  Fiscal  Year  2012  directs  DOD  to  take  a  number  of 
actions  consistent  with  our  recommendations,  with  the  aim  of  better 
assessing,  managing,  and  controlling  weapon  system  O&S  costs.8  In  light 
of  the  findings  from  our  prior  work,  you  requested  that  we  review  the  O&S 
cost  estimate  information  for  major  weapon  systems  that  DOD  submits  to 
Congress  within  the  annual  SARs.  Specifically,  our  objective  was  to 
determine  the  extent  to  which  the  SARs  provide  consistent  and  reliable 
O&S  cost  estimate  information  to  enable  effective  oversight  of  these 
weapon  system  costs. 

To  meet  our  objective,  we  reviewed  statutory  requirements  and  DOD 
guidance  for  reporting  weapon  system  O&S  cost  estimates  in  the  SARs. 
We  also  reviewed  DOD  cost-estimating  guidance,  which  identifies  the 
specific  categories  of  O&S  costs  for  weapon  systems,  and  GAO-identified 
cost-estimating  best  practices  to  identify  the  scope  and  nature  of  cost 
estimate  information  needed  for  effective  program  management  and 
oversight.  We  interviewed  DOD  and  military  service  officials  responsible 
for  weapon  system  acquisition,  logistics,  and  cost  analysis  to  understand 
DOD’s  approach  and  process  for  reporting  O&S  cost  estimates  in  the 
SARs. 

We  obtained  SARs  from  the  Defense  Acquisition  Management 
Information  Retrieval  system,  which  is  a  web-based  system  used  within 
DOD  to  collect  and  maintain  SAR  information  submitted  by  the  program 


6  Pub.  L.  No.  111-23  (2009). 

7GAO,  Defense  Management:  DOD  Needs  Better  Information  and  Guidance  to  More 
Effectively  Manage  and  Reduce  Operating  and  Support  Costs  of  Major  Weapon  Systems, 
GAO-10-717  (Washington,  D.C.:  July  20,  2010). 

8  Pub.  L.  No.  112-81,  §  832  (2011). 
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offices.  We  determined  that  the  data  in  this  system  accurately  reflected 
information  submitted  by  weapon  system  program  offices  and  therefore 
were  sufficiently  reliable  for  the  purposes  of  our  analysis.  We  obtained 
and  analyzed  annual  SAR  data  for  all  84  major  weapon  systems  that 
reported  O&S  costs  in  their  annual  SAR  for  2010.  This  analysis  covered 
the  time  period  from  2005  through  2010  with  the  exception  of  2008. 9  In 
addition,  from  these  84  weapon  systems,  we  selected  a  sample  of  15 
programs  for  further  analysis.10  Through  a  data  collection  instrument, 
interviews,  and  other  contacts,  we  obtained  information  from  the  program 
offices  on  the  O&S  cost  reports  they  had  submitted  in  the  SAR,  including 
cost  estimates,  where  available,  that  formed  the  basis  for  their  cost 
reports.  We  designed  the  sample  to  ensure  that  a  range  of  weapon 
systems  were  represented  based  on  commodity  type11  and  the  military 
service  under  which  the  system  acquisition  was  being  managed.  The 
results  from  this  nonprobability  sample  cannot  be  used  to  make 
inferences  about  all  major  weapon  systems  because  the  sample  may  not 
reflect  all  characteristics  of  the  population.  We  also  obtained  SARs  for  1 1 
weapon  systems  that  reported  an  annual  SAR  for  2010  but  did  not 
identify  O&S  costs  in  the  SAR.  We  obtained  information  from  the  program 
offices  about  the  reasons  these  costs  were  not  reported.  Our  scope  and 
methodology  is  discussed  further  in  appendix  I. 

We  conducted  this  performance  audit  from  February  201 1  to  February 
2012  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 


9  DOD  did  not  report  annual  SARs  for  2008.  In  addition,  because  some  of  the  weapon 
systems  began  reporting  SARs  at  some  point  during  this  time  period,  our  analysis  for  such 
systems  only  dates  back  to  their  initial  SAR  report. 

i  n 

Our  original  sample  included  16  weapon  systems.  However,  one  system,  the  Army’s 
Increment  1  Early-lnfantry  Brigade  Combat  Team,  was  subsequently  canceled. 

11  We  categorized  systems  as  aircraft,  ship,  ground,  or  “other”  (e.g.,  missile  programs  and 
command  and  control  systems). 
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Background 


Statutory  Requirements  for 
Submitting  SARs  to 
Congress 


Requirements  for  submitting  SARs  to  Congress,  including  the  timing  of 
these  reports  and  the  types  of  information  to  be  included,  are  established 
in  statute.  Under  10  U.S.C.  §  2432,  the  Secretary  of  Defense  shall  submit 
to  Congress  at  the  end  of  each  fiscal-year  quarter  a  report  on  current 
major  defense  acquisition  programs.12  Each  SAR  for  the  first  quarter  of  a 
fiscal  year  (also  known  as  the  comprehensive  annual  SAR)  shall  be 
designed  to  provide  to  the  Committee  on  Armed  Services  of  the  Senate 
and  the  Committee  on  Armed  Services  of  the  House  of  Representatives 
the  information  these  committees  need  to  perform  their  oversight 
functions.  The  comprehensive  annual  SAR  shall  be  submitted  within  60 
days  after  the  date  on  which  the  President’s  Budget  is  sent  to  Congress 
for  the  following  fiscal  year. 


The  statute  also  requires  that  the  annual  SAR  include  a  full  life-cycle  cost 
analysis  for  each  major  defense  acquisition  program  and  each  designated 
major  subprogram  included  in  the  report  that  is  in  the  system 
development  and  demonstration  stage  or  has  completed  that  stage.13 
Further,  the  Secretary  of  Defense  must  ensure  that  this  requirement  is 
implemented  in  a  uniform  manner,  to  the  extent  practicable,  throughout 
DOD.  The  term  full  life-cycle  cost,  with  respect  to  a  major  weapon 
system,  means  all  costs  of  development,  procurement,  military 
construction,  and  operations  and  support,  without  regard  to  funding 
source  or  management  control.  If  the  major  weapon  system  has  an 
antecedent  system,  a  full  life-cycle  cost  analysis  for  that  system  must  also 
be  reported.  The  SAR  reporting  requirement  ceases  after  90  percent  of 
the  items  are  delivered  or  90  percent  of  planned  expenditures  under  the 
program  are  made. 


12  SARs  for  the  second,  third,  and  fourth  quarters  of  a  fiscal  year,  known  as  quarterly 
SARs,  are  not  required  for  a  program  if  a  report  on  the  program  was  included  in  a 
previous  SAR  for  that  fiscal  year  and  during  the  period  since  that  report  the  program  has 
not  experienced  cost  increases  or  schedule  delays  beyond  certain  thresholds. 

This  stage  of  acquisition  is  now  called  engineering  and  manufacturing  development. 
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DOD’s  Guidance  for 
Implementing  the 
Statutory  SAR 
Requirements 


DOD  has  issued  various  guidance  documents  that  implement  the 
statutory  SAR  requirements.  This  guidance  is  contained  in  an  acquisition 
instruction,14  a  guidebook  on  defense  acquisition  best  practices,15  a  draft 
SAR  policy,16  and  an  annual  memorandum  on  preparing  SARs.17  DOD 
also  has  developed  instructions18  and  training  for  entering  SAR  data  into 
the  Defense  Acquisition  Management  Information  Retrieval  system.19 
According  to  officials,  program  offices  rely  on  DOD’s  implementation 
guidance  because  the  services  do  not  have  their  own  formal  SAR 
reporting  guidance.  DOD’s  guidance  is  summarized  below  and  discussed 
more  fully  in  appendix  II. 

DOD’s  implementation  guidance  states  that  SARs  should  be  submitted  at 
program  initiation  or  at  the  time  that  the  program  is  designated  as  a  major 
defense  acquisition  program,  and  then  at  least  annually  thereafter.  The 
guidance  requires  the  submission  of  a  quarterly  SAR  after  a  program  is 
rebaselined  based  on  a  major  milestone  decision.20  With  respect  to 
reporting  O&S  costs  in  the  SAR,  DOD’s  implementation  guidance  states 


14  DOD  Instruction  5000.02,  Operation  of  the  Defense  Acquisition  System  (Dec.  8,  2008). 

15  DOD,  Defense  Acquisition  Guidebook  (201 1 ). 

16  OSD,  Selected  Acquisition  Report  ( draft)  (2006).  Although  issued  in  draft,  the  policy 
was  never  issued  in  final  form. 

17  At  the  time  we  conducted  our  review,  the  most  recent  annual  guidance  was  issued  in 
January  2011.  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics, 
Memorandum  for  Assistant  Secretaries  of  the  Military  Services,  December  2010  Selected 
Acquisition  Reports  (SARs)  Guidance  (Jan.  14,  201 1 ). 

1 8 

Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics,  SAR  Data  Entry 
Instructions  (draft)  (Nov.  5,  2010). 

1 Q 

Throughout  the  report,  we  collectively  refer  to  all  of  DOD’s  SAR  guidance  as  DOD’s 
implementation  guidance,  including  the  draft  SAR  policy. 

on 

DOD’s  acquisition  process  includes  a  series  of  decision  milestones  as  the  systems 
enter  different  stages  of  development  and  production.  As  part  of  the  process,  the  DOD 
component  or  joint  program  office  responsible  for  the  acquisition  program  is  required  to 
prepare  life-cycle  cost  estimates,  which  include  O&S  costs,  to  support  these  decision 
milestones  and  other  reviews.  Key  decision  milestones  include  milestone  B,  which 
approves  entry  into  the  engineering  and  manufacturing  development  phase,  and 
milestone  C,  which  approves  entry  into  the  production  and  deployment  phase,  including 
low-rate  initial  production.  Continuation  into  full-rate  production  occurs  after  the  full-rate 
production  decision  review  is  held.  In  conjunction  with  a  milestone  decision,  a  program 
may  be  rebaselined,  which  means  that  the  cost,  quantity,  schedule,  and  performance 
goals  are  changed  to  reflect  the  current  status. 


Page  6 


GAO-12-340  Defense  Logistics 


that  program  offices  should  provide  explanatory  information  such  as  the 
source  and  date  of  the  cost  estimate,  assumptions  underlying  the 
estimate,  the  antecedent  system  used  for  comparison  purposes,  and  an 
explanation  of  how  average  annual  costs  were  calculated.  DOD  officials 
stated  that  programs  should  report  the  cost  estimate  that  was  developed 
for  the  latest  acquisition  milestone  decision.21  According  to  the  guidance, 
programs  should  report  total  estimated  O&S  costs  and  should  also  report 
average  O&S  costs  by  a  unit  of  measure  (e.g.,  average  annual  cost  per 
squadron,  average  annual  cost  per  system).22  DOD’s  guidance  states  that 
if  a  program  has  an  antecedent  system,23  then  O&S  costs  and 
assumptions  should  be  submitted  for  the  antecedent  system. 

In  addition  to  its  SAR  implementation  guidance,  DOD  has  issued 
guidance  for  developing  weapon  system  O&S  cost  estimates,  which 
provide  the  basis  for  the  O&S  cost  section  of  each  SAR.  Specifically,  the 
OSD  Cost  Analysis  Improvement  Group,  now  known  as  the  Cost 
Assessment  and  Program  Evaluation  (CAPE)  office,  has  established 
guidance  for  preparing  and  presenting  life-cycle  O&S  cost  estimates  at 
acquisition  milestone  reviews.24  O&S  cost  elements,  for  example,  are  to 
be  grouped  into  six  major  areas — unit-level  personnel,  unit  operations, 
maintenance,  sustaining  support,  continuing  system  improvements,  and 
indirect  support — which  are  further  broken  down  into  23  subelements.  In 
addition,  we  have  identified  federal  government  best  practices  for 
preparing  and  presenting  cost  estimates.  These  practices  include  tracking 
cost  estimates  over  time;  identifying  the  major  cost  drivers;  identifying  the 


21  Training  materials  from  the  Defense  Acquisition  University  similarly  reflect  that  the 
estimate  developed  for  the  latest  acquisition  milestone  should  be  reported  in  the  SAR. 
However,  the  draft  SAR  policy  differs  in  that  it  calls  for  reporting  the  “most  recent”  estimate 
of  life-cycle  O&S  costs. 

22  The  guidance  directs  program  offices  to  report  total  estimated  O&S  costs  in  constant 
dollars  as  well  as  then-year  dollars,  and  to  report  average  costs  in  constant  dollars. 
Constant  dollars  measure  the  value  of  purchased  goods  and  services  at  price  levels  which 
are  the  same  as  the  base  year  and,  unlike  then-year  dollars,  do  not  contain  any 
adjustments  for  inflationary  changes  that  have  occurred  or  are  forecasted  to  occur  outside 
the  base  year. 

23  An  “antecedent  system”  is  defined  by  DOD  as  one  that  has  been  replaced  by  another 
due  to  obsolescence  (technical  or  otherwise). 

24  OSD,  Operating  and  Support  Cost  Estimating  Guide  (October  2007). 
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method  and  process  for  estimating  each  cost  element;  and  comparing  the 
program-developed  cost  estimate  to  an  independent  cost  estimate.25 


DOD’s  Process  and  When  required,  a  comprehensive  annual  SAR  is  prepared  for  each  major 

Schedule  for  Preparing  weapon  system  by  the  program  office  that  is  managing  its  acquisition. 

SARs  Program  offices  are  responsible  for  weapon  systems  throughout  the  life 

cycle,  to  include  implementing,  managing,  and/or  overseeing  their 
development,  production,  fielding,  sustainment,  and  disposal.  The 
reporting  time  frame  for  the  annual  SAR  is  linked  to  the  issuance  of  the 
President’s  Budget,  which  occurs  early  in  the  calendar  year,  and  the  cost, 
schedule,  and  performance  data  reported  in  the  annual  SAR  should 
reflect  this  budget  request.26 

The  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics  begins  the  process  by  sending  out  its  annual  memorandum 
guidance  in  mid-January.  Program  offices  then  enter  data  into  the 
Defense  Acquisition  Management  Information  Retrieval  system  and 
submit  the  SARs  to  OSD  acquisition  officials,  generally  after  some  level  of 
internal  review  by  the  program  office,  the  Program  Executive  Officer,27 
and  the  military  service  under  which  the  program  is  organized.  OSD 
officials  review  the  SAR  submissions,  and  officials  within  the  Office  of  the 
Assistant  Secretary  of  Defense  (Logistics  and  Materiel  Readiness)  focus 
on  the  O&S  section  of  the  reports.  OSD  officials  then  hold  a  series  of 
meetings  with  the  services  and  program  office  representatives  to  discuss 
the  SAR  submissions  and  any  recommended  changes.  Consistent  with 
the  statutory  requirement,  the  final  annual  SAR  is  typically  submitted  to 
Congress  in  April,  60  days  after  the  President’s  Budget  has  been 
submitted  in  February. 


25  GAO,  GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
Managing  Capital  Program  Costs,  GAO-09-3SP  (Washington,  D.C.:  March  2009). 

26  The  annual  SAR  is  dated  at  the  end  of  the  calendar  year  that  just  ended  because  it  is 
the  first  quarterly  SAR  of  the  fiscal  year.  For  example,  the  annual  SAR  prepared  and 
reported  in  early  calendar  year  201 1  is  dated  December  2010. 

27  A  Program  Executive  Officer  is  the  military  or  civilian  official  who  has  responsibility  for 
directing  the  acquisition  of  several  major  weapon  systems,  as  well  as  other  acquisition 
programs.  This  individual  reports  to  and  receives  guidance  and  direction  from  the  service 
acquisition  executive. 
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DOD’s  Reports  to 
Congress  on 
Estimated  Weapon 
System  O&S  Costs 
Were  Inconsistent  and 
Sometimes  Unreliable 


Program  offices  reporting  life-cycle  O&S  cost  estimates  in  the  SAR  were 
often  inconsistent  in  their  cost  reporting  and  also  did  not  follow  best 
practices  for  presenting  cost  estimates.  In  addition,  some  programs  did 
not  provide  any  O&S  cost  estimates  in  the  2010  SAR.  Further,  several  of 
the  programs  we  reviewed  in  more  depth  reported  unreliable  O&S  cost 
data.  The  main  cause  for  these  deficiencies  was  a  lack  of  detailed  SAR 
implementation  guidance  for  reporting  O&S  costs.  In  addition,  DOD’s 
process  for  reviewing  the  O&S  cost  sections  of  the  SAR  prior  to  their  final 
submission  did  not  provide  assurance  that  the  program  offices  reported 
costs  uniformly,  to  the  extent  practicable,  and  that  these  reported  costs 
were  reliable.  In  the  absence  of  improvements  to  the  SAR  guidance  and 
to  DOD’s  review  process,  deficiencies  in  reporting  estimated  life-cycle 
O&S  costs  are  likely  to  continue.  Such  deficiencies  may  limit  visibility 
needed  for  effective  oversight  of  long-term  weapon  system  O&S  costs 
during  the  acquisition  process. 


Program  Offices  Were 
Inconsistent  in  Reporting 
Estimated  Life-Cycle  O&S 
Costs  and  Did  Not 
Incorporate  Best  Practices 
for  Presenting  These  Cost 
Estimates 


The  SAR  statute  requires  that  life-cycle  cost  reporting  for  major  weapon 
systems  be  uniform,  to  the  extent  practicable,  across  the  department,  but 
we  found  a  number  of  inconsistent  practices  in  how  program  offices  were 
reporting  life-cycle  O&S  cost  estimates  in  the  SAR.  Based  on  the  SAR 
submissions  we  reviewed,  program  offices  were  inconsistent  in  (1)  the 
explanatory  information  they  included  with  the  cost  estimates,  (2)  the 
source  of  the  cost  estimate  they  cited  as  the  basis  for  the  reported  costs, 
(3)  the  unit  of  measure  they  used  to  portray  average  costs,  (4)  the 
frequency  with  which  they  updated  reported  costs,  and  (5)  the  reporting  of 
antecedent  system  costs.  In  addition  to  these  inconsistencies,  we  found 
that  SAR  submissions  also  did  not  incorporate  best  practices  for 
presenting  cost  estimates,  such  as  tracking  cost  changes  over  time  and 
identifying  cost  drivers.  In  addition,  1 1  systems  did  not  provide  O&S  cost 
estimates  in  the  2010  SAR. 


Submitting  more  consistent  cost  reports  and  incorporating  best  practices 
for  presenting  cost  estimates  would  improve  visibility  of  estimated  life- 
cycle  O&S  costs  in  the  SAR,  as  decision  makers  will  have  more 
information  with  which  to  evaluate  the  reported  data.  For  example,  the 
inclusion  of  the  date  and  the  source  of  the  reported  estimate  provides 
context  about  the  currency  of  the  reported  costs  and  the  level  of  review 
(that  is,  whether  the  cost  estimate  was  prepared  by  the  program  office,  by 
the  military  service,  or  by  CAPE).  Likewise,  the  inclusion  of  significant 
assumptions  underlying  the  cost  estimate,  an  explanation  of  changes  in 
the  cost  estimate  from  the  prior  year,  and  information  on  major  cost 
drivers  provides  insight  into  the  cost  challenges  facing  the  program.  In 
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Explanatory  Information  Was 
Often  Minimal 


addition,  showing  average  costs  using  a  common  unit  of  measure  allows 
for  easier  comparison  of  program  costs  to  the  costs  of  similar 
commodities  (such  as  other  aircraft  programs). 

DOD’s  implementation  guidance  for  the  SAR  directs  programs  to  include 
explanatory  information  in  the  narrative  accompanying  the  O&S  cost 
estimates,  such  as  the  source  and  date  of  the  cost  estimate,  assumptions 
underlying  the  estimate  (such  as  operating  tempo,  expected  reliability  and 
maintainability  of  the  system,  maintenance  concept,  and  manning  and 
logistics  policies),  the  antecedent  program  used  for  comparison  purposes, 
and  an  explanation  of  how  average  costs  were  calculated.  Although 
explanatory  information  can  provide  context  and  background  for 
understanding  reported  costs,  we  found  that  the  explanatory  information 
included  in  the  O&S  narrative  was  often  minimal. 

Of  the  84  programs  that  reported  O&S  costs  in  the  2010  SAR,  we  found 
that  35  (42  percent)  did  not  include  the  source  of  the  estimate  and  12  (14 
percent)  did  not  include  the  date  of  the  estimate  in  the  O&S  narrative. 
Additionally,  for  the  15  programs  in  our  sample,  we  found  that  beyond 
providing  a  few  basic  details  such  as  the  number  of  units  that  were  to  be 
acquired,  their  expected  service  life,  and  operating  tempo,  where 
applicable,  the  O&S  narrative  contained  minimal  explanation  of  reported 
cost  estimates  and  the  assumptions  underlying  these  estimates,  as  the 
following  examples  illustrate: 

•  The  program  office  for  the  Army’s  High  Mobility  Artillery  Rocket 
System  included  several  assumptions,  such  as  the  number  of 
launchers  and  the  service  life.  However,  instead  of  reporting 
additional  O&S  cost  estimate  assumptions  (such  as  operating  tempo 
and  expected  reliability/maintainability)  in  the  SAR  narrative,  the 
program  stated  that  this  information  was  available  in  the  service  cost 
estimate. 

•  The  Joint  Mine  Resistant  Ambush  Protected  (MRAP)  vehicle  program 
office  noted  a  few  specific  assumptions,  such  as  the  expected  service 
life  of  the  fleet  and  the  cost  per  mile  for  replenishing  spare  parts. 
However,  the  remaining  O&S  narrative  for  the  program  generically 
explained  that  the  estimate  included  personnel,  training,  facilities, 
vehicle  and  component  repair,  and  sustainment  overhauls,  but 
provided  no  other  specifics  on  these  areas. 

•  Only  three  programs — the  Army’s  High  Mobility  Artillery  Rocket 
System,  the  Air  Force  Joint  Primary  Aircraft  Training  System,  and  the 
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Army’s  Force  XXI  Battle  Command  Brigade  and  Below  (FBCB2) — 
included  the  maintenance  concept  planned  for  that  system  in  their 
O&S  narratives.  However,  even  in  these  cases,  the  explanatory 
information  for  O&S  costs  was  very  limited. 

•  None  of  the  15  programs  included  assumptions  on  the  reliability  and 
maintainability  of  the  weapon  system  in  their  O&S  narrative. 

While  not  required  by  DOD’s  implementation  guidance,  1  of  the  15 
programs  in  our  sample  included  explanatory  information  on  cost  drivers 
in  the  SAR  O&S  narrative.  The  V-22’s  SAR  submission  for  2009  provided 
an  explanation  of  the  significant  O&S  cost  increase  from  the  prior  SAR  in 
2007.  In  the  2007  SAR,  the  total  O&S  costs  reported  were  $48.8  billion 
(fiscal  year  2005  dollars).  In  the  2009  SAR,  the  program  reported  that  this 
amount  had  grown  to  $75  billion  (fiscal  year  2005  dollars),  and  that  the 
O&S  cost  category  showing  the  greatest  increase  was  unit-level 
consumption.28  In  the  O&S  narrative,  the  program  office  attributed  the 
majority  of  the  cost  increase  to  changes  in  the  methodology  used  to 
estimate  unit-level  consumption  costs.  Specifically,  the  estimate  was 
updated  with  the  actual  costs  of  parts  from  fiscal  year  2009  and  with 
projected  future  cost  growth  for  parts  higher  than  OSD’s  inflation 
indices.29  The  program  office  also  noted  actions  being  taken  to  reduce 
unit-level  consumption  costs,  such  as  changes  to  contracting  strategy  and 
accelerated  timelines  for  repair  capabilities. 

GAO-identified  best  practices  for  presenting  cost  estimates  include 
identifying  the  largest  cost  elements  and  cost  drivers,  and  providing 
enough  information  for  informed  decision  making.  In  addition,  we  have 
previously  reported  that  leading  companies  identify  major  drivers  of  O&S 
costs  and  work  with  manufacturers  to  reduce  these  costs.30  During  our 


po 

Unit-level  consumption  is  an  O&S  cost  category  that  includes  a  number  of  subelements 
such  as  the  cost  of  fuel  and  energy  resources;  operations,  maintenance,  and  support 
materials  consumed  at  the  unit  level;  depot-level  reparable  spares  (individual  parts, 
subassemblies,  or  assemblies)  required  for  recurring  repair;  operational  munitions 
expended  in  training;  transportation  in  support  of  deployed  unit  training;  temporary 
additional  duty/temporary  duty  pay;  and  other  unit-level  consumption  costs,  such  as 
purchased  services  (e.g.,  equipment  leases  and  service  contracts). 

pq 

OSD’s  inflation  indices  are  published  annually  and  are  used  to  develop  cost  estimates. 

on 

GAO,  Best  Practices:  Setting  Requirements  Differently  Could  Reduce  Weapon 
Systems’  Total  Ownership  Costs,  GAO-03-57  (Washington,  D.C.:  Feb.  11, 2003). 
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Sources  of  Reported  Cost 
Estimates  Varied  among 
Programs 


current  review,  we  found  that  programs  typically  use  CAPE’s  O&S  cost 
element  structure  in  reporting  O&S  costs,  but  their  presentation  is  limited 
to  the  six  major  elements  (e.g.,  unit-level  personnel,  maintenance,  indirect 
support).  Since  each  major  O&S  cost  element  includes  various  costs,  this 
information  is  not  sufficient  to  identify  specific  cost  drivers.  Using  lower- 
level  cost  elements,  as  provided  for  in  CAPE’s  cost  element  structure, 
could  provide  greater  visibility  of  O&S  costs  for  oversight  by  decision 
makers.  For  example,  as  noted  in  the  case  of  the  V- 22  discussed  above, 
the  unit-level  consumption  cost  element  consists  of  a  number  of 
subelements  that  can  provide  additional  insight  into  the  discrete  factors 
driving  a  change  in  the  estimated  life-cycle  O&S  costs  for  that  system. 

Various  cost  estimates  may  be  developed  over  the  life  cycle  of  a  weapon 
system,  and  DOD  officials  stated  that  programs  should  report  the  cost 
estimate  developed  for  the  latest  acquisition  milestone  decision.  We 
found  that  program  offices — when  a  source  was  cited — cited  several 
different  sources  as  the  basis  for  their  reported  O&S  cost  information  in 
the  2010  SAR,  and  they  did  not  provide  an  explanation  for  selecting  the 
source  that  was  used  rather  than  another  source  that  may  have  been 
available.  As  shown  in  table  1 ,  for  the  84  programs  that  included  O&S 
costs  in  the  2010  SAR,  42  (50  percent)  of  the  programs  cited  a  specific 
cost  estimate  as  the  source  of  reported  O&S  costs.  These  sources  were 
either  a  program  office  cost  estimate,  service  cost  estimate,  or  CAPE 
independent  cost  estimate.  Another  35  programs  (42  percent)  did  not  cite 
a  source,  as  previously  noted.  The  remaining  7  programs  (8  percent) 
cited  a  source  other  than  a  specific  cost  estimate. 


Table  1:  Source  of  Cost  Estimate  Cited  for  84  Programs  Reporting  O&S  Costs  in  the 
2010  SAR 


Source  cited  in  SAR 

No  source  cited 
Program  office  cost  estimate 
Service  cost  estimate 
Other 

CAPE  independent  cost  estimate 

Total 


Source:  GAO  analysis  of  2010  SAR  data. 


Number  of  programs 

35 

23 

15" 

7 

4 

84 


Five  programs  in  the  “other”  category  in  table  1  referred  to  cost  estimates 
but  did  not  provide  enough  detail  to  determine  what  type  of  cost  estimate 
was  used.  For  example,  one  program  cited  a  “validated  cost  estimate” 
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without  additional  specificity  about  this  estimate.  Similarly,  four  programs 
stated  only  that  the  source  of  their  SAR  O&S  costs  was  a  cost  estimate 
prepared  for  an  acquisition  decision,  but  they  did  not  provide  additional 
information  to  identify  a  specific  cost  estimate.  The  remaining  two 
programs  in  the  “other”  category  in  table  1,  both  of  which  were  included  in 
our  sample,  cited  a  source  other  than  a  cost  estimate.  One  of  these 
programs,  the  Navy  Multiband  Terminal,  reported  total  costs  from  the 
milestone  C  acquisition  program  baseline,31  despite  the  existence  of  a 
service  cost  estimate  prepared  for  the  acquisition  decision  in  July  2010. 
The  other  program,  the  Air  Force’s  Navstar  Global  Positioning  System 
(GPS),  reported  using  current  and  future  funding  data  instead  of  a  cost 
estimate. 

Overall,  six  programs  in  our  sample  were  among  those  that  did  not  cite 
the  source  of  the  estimate  used  to  report  O&S  costs.  When  we  asked 
these  six  programs  what  source  was  used,  five  stated  that  the  O&S  cost 
estimate  data  in  the  2010  SAR  were  derived  from  program  office  cost 
estimates,  and  the  remaining  program  office  stated  that  the  source  was  a 
CAPE  independent  cost  estimate.  The  other  nine  programs  in  our  sample 
had  cited  a  source,  with  five  citing  a  program  office  cost  estimate;  two 
citing  a  service  cost  estimate;  and  two,  as  noted  above,  citing  either  an 
acquisition  program  baseline  or  funding  data  as  the  source  of  their  O&S 
costs. 

As  shown  in  table  1 ,  some  programs  cited  a  CAPE  independent  cost 
estimate  as  the  source  of  the  O&S  costs  reported  in  the  2010  SAR. 
However,  we  found  that  one  program  in  our  sample,  the  LHA  6  America 
Class,  cited  a  program  office  cost  estimate  even  though  CAPE  had 
developed  an  independent  cost  estimate.  Further,  while  not  required,  the 
program  did  not  mention  in  the  SAR  that  an  independent  cost  estimate 
had  been  developed.  Since  2005  the  LHA  6  America  Class  program  has 
reported  total  O&S  costs  of  $4.45  billion  (fiscal  year  2006  dollars)  in  its 
SAR  submissions,  reflecting  a  2005  program  office  cost  estimate. 
However,  CAPE’s  2006  independent  cost  estimate  of  the  program’s  O&S 
costs  was  about  $300  million  (7  percent)  higher.  According  to  a  CAPE 
memorandum,  this  higher  estimate  was  also  not  adjusted  for  cost  growth 
above  inflation.  CAPE  noted  that  O&S  costs  for  the  LHA  1 ,  an  antecedent 


31  An  acquisition  program  baseline  quantifies  key  parameters  for  the  performance,  cost, 
and  schedule  of  a  program  throughout  the  acquisition  phase. 
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Units  of  Measure  for  Reporting 
Average  Costs  Were  Not 
Consistent 


system,  had  increased  4  percent  annually  since  1990  due  to  increased 
mission  personnel  and  overhaul  costs.  According  to  CAPE,  adjusting  for 
this  same  rate  of  cost  growth  above  inflation  in  its  LHA  6  estimate  would 
result  in  an  additional  $530  million  throughout  the  system’s  life  cycle,  or 
total  O&S  costs  of  $5.29  billion.  Additionally,  F-35  officials  told  us  that 
they  plan  to  continue  using  the  program  office’s  cost  estimate  to  report 
O&S  costs  in  the  SAR  although  CAPE  is  preparing  an  independent  cost 
estimate  for  the  program’s  next  acquisition  milestone. 

GAO-identified  best  practices  for  presenting  cost  estimates  include 
providing  a  comparison  of  the  program  estimate  to  an  independent  cost 
estimate,  with  an  explanation  of  results  and  differences.  Such  a 
comparison  is  beneficial  because  an  independent  cost  estimate  should 
provide  an  objective  and  unbiased  assessment  of  expected  program 
costs  that  tests  the  program’s  estimate  for  reasonableness.  History  has 
shown  a  pattern  of  higher,  more  accurate  cost  estimates  the  further  away 
from  the  program  office  the  independent  cost  estimate  is  prepared.  In  the 
2009  Weapon  Systems  Acquisition  Reform  Act,  Congress  placed  greater 
emphasis  on  independent  review  of  program  cost  estimates32  by  requiring 
that  CAPE  review  cost  estimates  prepared  in  connection  with  all  major 
weapon  systems,  and  conduct  independent  cost  estimates  for  certain 
systems  prior  to  the  milestone  A,  milestone  B,  low-rate  initial  production, 
and  full-rate  production  acquisition  decisions.  Prior  to  the  Act,  CAPE  was 
required  to  conduct  independent  cost  estimates  for  some  programs,  but 
was  not  required  to  review  cost  estimates  prepared  for  all  major  weapon 
systems. 

DOD’s  implementation  guidance  for  the  SAR  states  that  programs  should 
report  average  O&S  costs  in  a  unit  of  measure  determined  by  the  military 
service  under  which  the  system’s  acquisition  is  being  managed. 

Programs  are  to  report  these  average  costs  using  CAPE’s  major  cost 
elements.  We  found  that  several  program  offices  had  changed  the  unit  of 
measure  they  reported  in  the  SAR  from  that  used  in  previous  SARs.  In 
addition,  we  found  that  the  units  of  measure  that  were  being  reported 
varied,  particularly  among  aircraft  programs.  These  inconsistencies  make 
it  difficult  to  compare  a  program’s  current  and  prior-year  costs,  or  to 
compare  costs  of  similar  programs. 


op 

According  to  our  Cost  Estimating  and  Assessment  Guide,  an  independent  cost  estimate 
is  the  most  rigorous  form  of  independent  review;  however,  other  independent  reviews  are 
also  useful  for  decision  making. 
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Of  the  84  programs  that  reported  O&S  costs  in  the  2010  SAR,  5  (6 
percent)  changed  the  average  unit  of  measure  reported  from  that  used 
the  prior  year.  Specifically,  two  aircraft  programs  went  from  reporting 
costs  per  squadron  in  the  2009  SAR  to  reporting  costs  per  aircraft  in  their 
2010  SAR,  a  missile  program  went  from  reporting  costs  per  unit  in  the 
2009  SAR  to  reporting  total  program  costs  in  the  2010  SAR,  and  two 
programs  for  communications  systems  went  from  reporting  total  program 
costs  in  the  2009  SAR  to  reporting  costs  per  radio  in  the  2010  SAR. 
These  last  two  programs —  Joint  Tactical  Radio  System  (JTRS)  Ground 
Mobile  Radios  and  JTRS  Handheld,  Manpack,  and  Small  Form  Fit — were 
included  in  our  sample.  When  we  asked  why  they  changed  the  unit  of 
measure,  program  officials  responded  that  the  decision  was  made  based 
on  feedback  they  received  from  OSD  when  their  2010  SAR  submissions 
were  undergoing  review.  Of  the  5  programs,  only  the  two  aircraft 
programs  disclosed  in  the  SAR  that  the  unit  of  measure  for  that  system 
had  changed  from  the  prior  year.  These  two  programs  reported  that  they 
changed  the  unit  of  measure  in  order  to  standardize  the  calculation  and 
increase  the  comparability  of  programs  within  the  same  major  command. 

Also,  based  on  analysis  of  the  84  systems,  we  found  the  most  variation  in 
the  unit  of  measure  among  aircraft  systems.  Different  programs  reported 
the  average  cost  per  flying  hour33  or  the  average  annual  cost  per  aircraft, 
per  squadron  of  aircraft,  or  per  the  entire  fleet.  This  issue  was  also 
evident  among  the  programs  in  our  sample  that  we  analyzed  in  more 
depth.  For  example,  the  F-35  program  reported  average  cost  per  flying 
hour,  the  V- 22  program  reported  average  cost  per  aircraft,  the  F-22 
program  reported  average  cost  per  squadron,  and  the  Joint  Primary 
Aircraft  Training  System  (a  training  aircraft)  reported  average  cost  for  the 
whole  fleet.  Ship  costs,  in  contrast,  were  generally  reported  as  average 
cost  per  ship  or  hull,  although  one  ship  program  reported  average  annual 
cost  per  fleet.  O&S  costs  for  ground  and  other  types  of  weapon  systems 
were  usually  reported  as  either  cost  per  weapon  system  unit  or  total  cost 
for  all  weapon  system  units.  However,  a  few  other  metrics  were  reported 
by  these  programs,  such  as  average  annual  cost  per  battalion  or  per 
brigade  combat  team. 


oo 

With  regard  to  cost  per  flying  hour,  DOD  officials  explained  that  there  are  numerous 
formulas  used  throughout  DOD.  Thus,  while  several  aircraft  system  program  offices 
reported  average  cost  per  flying  hour,  they  may  not  have  used  comparable  data  and 
calculation  methodologies. 
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Cost  Estimates  Were  Updated 
at  Different  Intervals 


Although  portraying  average  costs  with  a  unit  of  measure  could  be  useful 
for  tracking  cost  changes  over  time,  we  found  that  it  was  generally  not 
possible  to  identify  changes  in  estimated  O&S  costs  based  on  the 
information  reported  in  a  single,  annual  SAR,  since  programs  do  not 
report  costs  from  the  prior  SAR.  Although  major  weapon  system 
programs  are  required  to  identify  and  reconcile  changes  to  estimated 
acquisition  costs  from  the  prior  SAR,  and  to  provide  an  explanation  for 
each  change,  this  is  not  required  for  O&S  costs.  Even  though  two  of  our 
sample  programs,  the  V-22  and  the  Navstar  GPS,  included  a  statement  in 
the  SAR  narrative  that  their  O&S  costs  had  changed,  it  was  not  possible 
to  tell  how  much  without  the  prior  year’s  cost  data.  Our  year-to-year 
comparisons  of  reported  costs  in  the  SARs  showed  that  cost  changes 
were  occurring.  For  example,  we  found  that  the  total  estimated  O&S  costs 
for  the  JTRS  Handheld,  Manpack,  and  Small  Form  Fit  program 
decreased  from  $25.5  billion  (fiscal  year  2004  dollars)  in  2009  to  $10.2 
billion  in  2010  (fiscal  year  2004  dollars).  This  $15.3  billion  decrease 
occurred  despite  an  increase  in  acquisition  quantity  of  about  5,000  radios, 
from  around  216,000  to  around  221,000.  This  change,  as  well  as  the 
reasons  for  the  change,  was  not  identified  in  the  SAR  narrative.  Similarly, 
we  found  that  the  total  estimated  O&S  costs  for  the  F-35  program 
increased  $50  billion  (fiscal  year  2002  dollars)  from  2009  to  2010.  The 
reason  for  this  increase  was  not  explained  in  the  O&S  narrative  in  the 
SAR. 

According  to  GAO-identified  best  practices  for  presenting  cost  estimates, 
cost  estimates  should  be  tracked  over  time.  Specifically,  after  an  estimate 
is  updated,  a  comparison  of  the  current  and  prior  estimate  should  be 
routinely  performed  and  documented,  and  the  results  reported  to  decision 
makers.  A  documented  comparison  allows  cost  estimators  to  see  how 
well  they  are  estimating  and  how  the  program  is  changing  over  time.  It 
also  allows  others  to  track  the  estimates  and  to  identify  when,  by  how 
much,  and  why  the  program  cost  more  or  less  than  planned. 

Updated  cost  estimates  can  help  to  ensure  that  decision  makers  have  the 
most  current  data  available  on  a  program.  The  SAR  statute  requires 
major  defense  acquisition  programs  to  begin  reporting  when  the  program 
is  approved  to  begin  the  development  phase  of  the  acquisition  process  at 
milestone  B,  and  DOD’s  implementation  guidance  similarly  states  that  a 
SAR  should  first  be  submitted  when  a  program  is  initiated,  normally  at 
milestone  B,  or  designated  as  a  major  defense  acquisition  program,  and 


Page  16 


GAO-12-340  Defense  Logistics 


also  when  the  program  is  rebaselined  after  a  major  milestone  decision. 
DOD  officials  stated  that  programs  should  report  the  cost  estimate 
developed  for  the  latest  acquisition  milestone  decision.  34  Our  analysis  for 
the  84  major  weapon  system  programs  that  included  O&S  costs  in  the 
2010  SAR  showed  that  program  offices  were  inconsistent  in  the 
frequency  of  their  O&S  cost  updates  between  2005  and  2010. 35  In  many 
cases,  programs  provided  more  frequent  updates  than  required  by  DOD’s 
guidance,  sometimes  annually.  However,  8  (13  percent)  of  the  61 
programs  that  were  included  in  the  SAR  every  year  during  the  2005  to 
2010  period  did  not  update  their  O&S  costs  at  any  time  during  that  period. 
In  contrast,  47  programs  (56  percent)  of  the  84  programs  in  the  2010 
SAR  reported  using  a  cost  estimate  that  was  prepared  in  2010  or  201 1  as 
the  source  of  their  O&S  costs.36  These  included  7  programs  that  began 
reporting  SARs  in  2009  or  2010. 

Of  the  1 5  programs  in  our  sample,  3  did  not  update  their  SAR  O&S  costs 
during  the  period  between  2005  and  2010,  5  updated  their  costs  once,  5 
updated  their  costs  2  or  3  times,  and  2  updated  their  O&S  costs  4  times 
during  the  period.37  For  example,  the  Navy’s  LHA  6  America  Class 
program  office  has  consistently  reported  the  O&S  costs  estimated  for 
milestone  B,  the  program’s  only  acquisition  milestone  while  under  SAR 
reporting  requirements,  in  the  annual  SARs  since  2005.  Program  officials 
told  us  that  they  were  in  the  process  of  developing  a  new  cost  estimate 
for  the  LHA  7,  the  next  ship  in  the  America  Class,  and  planned  to  use  the 
new  estimate  as  the  source  to  report  O&S  costs  in  the  program’s  201 1 
SAR  submission,  if  complete.  Also,  the  Army’s  FBCB2  program  has  not 


34  Training  materials  from  the  Defense  Acquisition  University  similarly  reflect  that  the 
estimate  developed  for  the  latest  acquisition  milestone  should  be  reported  in  the  SAR. 
However,  as  noted  earlier,  the  draft  SAR  policy  differs  in  that  it  calls  for  reporting  the  “most 
recent”  estimate  of  program  O&S  costs. 

35  As  noted  earlier,  a  number  of  programs  did  not  include  the  date  of  the  cost  estimate 
used  in  the  December  2010  SAR. 

36  Because  the  annual  SAR  reporting  process  occurs  in  the  calendar  year  after  the  date  of 
the  SAR,  a  cost  estimate  that  was  completed  in  early  201 1  could  be  used  to  report  costs 
in  the  2010  SAR. 

37  One  of  the  programs  that  did  not  update  O&S  costs  between  2005  and  2010  only 
started  reporting  annual  SARs  in  2009  and  did  not  update  the  costs  in  2010.  Also,  one  of 
the  programs  that  updated  annually — the  Navstar  GPS — reported  the  program’s  current 
funding  and  projected  requirements  for  an  8-year  period  instead  of  a  life-cycle  O&S  cost 
estimate. 
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updated  its  O&S  SAR  costs  and  is  reporting  costs  estimated  in  2004, 
even  though  the  program’s  production  quantity  has  quadrupled  since 
then.38  FBCB2  program  officials  told  us  that  since  its  full-rate  production 
decision  in  2004,  the  program  has  experienced  nearly  continuous 
changes  to  its  production  quantity  requirement,  resulting  in  a  significant 
effort  to  maintain  and  update  the  acquisition  portion  of  the  cost  estimates 
and  little  time  to  research  and  update  the  O&S  portion  of  the  cost 
estimates. 

In  contrast,  several  of  our  sample  programs  updated  their  O&S  costs 
annually.  The  F-35  program  has  updated  the  reported  SAR  O&S  costs 
annually  since  2006,  the  beginning  of  the  period  we  reviewed.  According 
to  F-35  program  officials,  they  chose  to  do  this  because  the  F-35  is  a 
high-visibility,  high-interest  program.  Further,  estimating  O&S  costs 
annually  helps  inform  DOD  leadership  and  keeps  partner  countries 
updated,  program  officials  noted.  Additionally,  the  Joint  MRAP  program 
office  has  updated  its  SAR  O&S  costs  annually  since  the  program  began 
reporting  these  costs  in  2009  and  plans  to  do  so  until  the  services 
assume  responsibility  for  the  system  around  2013.  Program  officials  said 
they  are  incorporating  actual  cost  data  from  the  field  as  it  becomes 
available  and  updating  O&S  costs  annually  in  order  to  give  the  services 
the  best  data  once  the  transfer  takes  place.  Finally,  the  Army’s  MQ-1C 
program  has  updated  its  SAR  O&S  costs  annually  since  2009.  Although 
these  costs  were  updated  in  2010  for  several  reasons,  including  an 
increase  in  the  number  of  systems  to  be  acquired,  program  officials  said 
they  do  not  plan  to  update  the  program’s  O&S  costs  annually. 

Officials  for  the  remaining  programs  in  our  sample,  which  updated  their 
O&S  costs  intermittently,  gave  various  reasons  for  updating  their 
program’s  SAR  O&S  costs  when  they  did.  While  one  program  updated 
the  SAR  as  required  to  reflect  the  O&S  costs  estimated  for  an  acquisition 
decision,  other  programs  in  our  sample  chose  to  update  the  costs  after 
they  developed  estimates  to  reflect  changes  to  the  acquisition  program 
(e.g.,  changes  in  production  quantity  or  schedule),  to  incorporate  actual 
O&S  costs  that  are  considerably  different  than  previously  estimated,  or  to 
comply  with  guidance  not  related  to  the  SAR.  For  example,  the  Navy’s 


38  The  O&S  costs  reported  in  the  FBCB2’s  2006  SAR  are  different  than  those  reported  in 
the  2005  SAR.  However,  program  officials  told  us  that  this  was  a  mistake  and  the  costs 
had  not  actually  been  updated.  This  case  is  discussed  further  later  in  our  report. 
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Antecedent  Program  Costs 
Were  Often  Not  Reported 


V-22  program  office  updated  the  O&S  costs  in  the  2009  SAR  because 
actual  O&S  costs  incurred  after  the  program’s  initial  operational  capability 
in  2007  for  the  Marine  Corps  and  2009  for  the  Air  Force  were  significantly 
higher  than  had  been  anticipated  in  the  program’s  most  recent  cost 
estimate.  Prior  to  the  2009  update,  the  V-22  was  reporting  costs  based  on 
the  estimate  completed  for  an  acquisition  decision  in  2005.  The  V-22 
program  office,  in  conjunction  with  U.S.  Naval  Air  Systems  Command, 
plans  to  review  the  program’s  O&S  costs  annually  and  update  the  SAR  as 
necessary  until  the  program  stops  reporting  SARs.  According  to  officials, 
the  final  deliveries  of  the  V-22  are  scheduled  for  2020.  As  another 
example,  the  Joint  Primary  Aircraft  Training  System  program  updated 
O&S  costs  in  the  2010  SAR  after  reporting  the  same  costs  since  2001 . 
According  to  officials,  an  updated  program  office  cost  estimate  was 
developed  to  comply  with  a  policy  from  the  program’s  major  command 
that  cost  estimates  be  updated  annually. 

DOD  acquisition  best  practices  and  GAO-identified  cost-estimating  best 
practices  call  for  maintaining  updated  estimates  of  program  costs. 
According  to  the  Defense  Acquisition  Guidebook,  although  a  DOD  or 
service  cost  estimate  is  required  at  milestone  reviews,  it  is  a  good 
practice  for  this  estimate,  or  at  least  its  underlying  program  office  cost 
estimate,  to  be  updated  more  frequently,  usually  annually.  Updated 
estimates  should  be  useful  in  program  management  and  financial 
management  throughout  the  life  of  the  program.  GAO-identified  best 
practices  call  for  continual  updates  of  cost  estimates  to  keep  them 
relevant  and  current,  as  most  programs  do  not  remain  static,  especially 
those  in  development.  Routine  updates  that  incorporate  actual  data  result 
in  higher-quality  estimates  as  the  program  matures.  Further,  updating  the 
cost  estimate  provides  an  accuracy  check,  defense  of  the  estimate  over 
time,  shorter  estimate  preparation  times,  and  archived  cost  and  technical 
data  for  use  in  future  estimates. 

In  accordance  with  the  SAR  statute,  DOD’s  implementation  guidance 
states  that  if  a  program  has  an  antecedent  system,  then  O&S  costs  and 
assumptions  should  be  submitted  for  the  antecedent  system.  We  found 
that  program  offices,  however,  were  inconsistent  in  reporting  on 
antecedent  system  costs,  with  many  not  reporting  any  O&S  cost  data. 
Specifically,  57  (68  percent)  of  the  84  programs  reporting  O&S  costs  in 
the  2010  SAR  did  not  report  O&S  costs  for  an  antecedent  system.  It  was 
unclear  from  the  SARs  how  program  offices  had  identified  an  antecedent 
system  or  whether,  in  cases  where  no  antecedent  system  costs  were 
included,  the  program  offices  had  determined  that  an  antecedent  system 
did  not  exist. 
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Some  Programs  Did  Not  Report 
Any  O&S  Costs 


Nine  of  the  1 5  programs  in  our  sample  did  not  report  O&S  costs  for  an 
antecedent  system  in  the  2010  SAR.  Officials  from  these  program  offices 
provided  various  reasons  for  not  reporting  antecedent  system  costs, 
including  that  the  system  was  the  first  of  its  type  or  not  intended  to 
replace  any  other  system,  that  the  system  had  advanced  capabilities  or 
no  other  system  was  similar  enough  for  comparison,  and  that  the  system 
was  replacing  several  legacy  systems.  As  an  example,  Joint  MRAP 
program  officials  said  other  systems,  such  as  the  High  Mobility 
Multipurpose  Wheeled  Vehicle,  were  too  different  for  cost  comparisons. 
As  another  example,  the  Navy  Multiband  Terminal  program  began 
reporting  in  the  2006  SAR  and  has  never  reported  antecedent  O&S  costs. 
According  to  program  officials,  an  antecedent  system  was  not  identified 
because  the  system  was  replacing  several  legacy  weapon  systems. 
However,  during  a  joint  OSD/Navy  SAR  review  meeting  in  March  201 1 , 
the  program  office  was  instructed  to  list  two  systems  as  antecedent 
systems  in  the  2010  SAR.  While  the  program  identified  the  Super  High 
Frequency  and  Navy  Extremely  High  Frequency  Satellite  programs  as 
antecedent  systems  in  the  O&S  section  of  its  SAR,  it  also  stated  that 
program  costs  for  these  systems  were  not  readily  available. 

The  SAR  statute  requires  that  all  program  costs  be  reported,  regardless 
of  funding  source  or  management  control.  However,  we  found  that  of  the 
95  major  weapon  systems  that  had  passed  milestone  B  and  reported 
costs  in  the  2010  SAR,39 11  (12  percent)  did  not  identify  any  O&S  costs  in 
their  SARs.  The  1 1  programs,  as  of  December  2010,  accounted  for  a  total 
estimated  investment  of  $56.7  billion  (fiscal  year  201 1  dollars)  for 
research  and  development,  procurement,  military  construction,  and 
acquisition-related  operation  and  maintenance  (see  table  2).  Most  of  the 
programs  that  did  not  report  O&S  costs  were  modifications  to  other 
weapon  systems  but  qualify  as  major  defense  acquisition  programs 
based  on  their  procurement  or  research  and  development  costs. 


oq 

One  program  submitted  a  2010  SAR  but  had  not  yet  passed  acquisition  milestone  B. 
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Table  2:  Major  Weapon  System  Programs  That  Did  Not  Identify  O&S  Costs  in  Their  2010  SARs 

Dollars  in  billions 

Major  weapon  system 

Service 

Estimated  investment  costs3 

C-5  Avionics  Modernization  Program 

Air  Force 

$1.3 

C-5  Reliability  Enhancement  and  Reengineering  Program 

Air  Force 

$7.3 

GPS  III 

Air  Force 

$4.3 

Multi-Platform  Radar  Technology  Insertion  Program 

Air  Force 

$1.4 

Airborne  Signals  Intelligence  Payload 

Air  Force 

$0.5 

Large  Aircraft  Infrared  Countermeasures 

Air  Force 

$0.5 

B-2  Radar  Modernization  Program 

Air  Force 

$1.3 

B-2  Extremely  High  Frequency  Satellite  Communications 

Air  Force 

$0.6 

Apache  Block  NIB 

Army 

$2.2 

Chemical  Demilitarization-Assembled  Chemical  Weapons  Alternatives 

DOD 

$10.0 

Chemical  Demilitarization-U.S.  Army  Chemical  Materials  Agency 

Army 

$27.4 

Source:  GAO  analysis  of  2010  SAR  data. 

aCosts  are  in  fiscal  year  201 1  dollars  and  include  costs  associated  with  research  and  development, 
procurement,  military  construction,  and  acquisition-related  operation  and  maintenance. 


Eight  of  the  programs  that  did  not  report  O&S  costs  are  major 
modifications  to,  or  subsystems  of,  Air  Force  weapon  systems.40  When 
we  asked  why  O&S  costs  were  not  reported,  officials  from  six  Air  Force 
programs  said  they  did  not  report  O&S  costs  in  the  2010  SAR  because 
they  do  not  fund  or  track  these  costs.  For  example,  officials  for  two 
programs  associated  with  the  C-5  aircraft  explained  that  all  O&S  fleet 
costs  are  the  responsibility  of  another  entity,  the  System  Program 
Manager  at  Warner  Robins  Air  Logistics  Center  in  Georgia.  Program 
officials  for  the  other  two  Air  Force  programs,  the  B-2  Radar 
Modernization  Program  and  B-2  Extremely  High  Frequency  Satellite 
Communications  program,  told  us  that  these  modification  programs  were 
expected  to  reduce  O&S  costs  and  they  could  not  input  cost  reductions 
into  DOD’s  Defense  Acquisition  Management  Information  Retrieval 
system,  the  database  that  maintains  SAR  data.  In  contrast  to  these 
modification  programs,  the  Air  Force’s  C-130  Avionics  Modernization 
Program  did  report  total  estimated  O&S  costs  in  the  2010  SAR. 


40  These  modifications  are  considered  major  defense  acquisition  programs  in 
development  and  are  therefore  required  to  report  submit  SARs. 
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According  to  officials,  one  of  the  remaining  three  programs — the  Army’s 
Apache  Block  NIB — was  not  required  to  report  O&S  costs  in  the  SAR,  as 
approved  by  the  Defense  Acquisition  Executive.41  The  other  two 
programs  are  the  Chemical  Demilitarization-Assembled  Chemical 
Weapons  Alternatives,  and  the  Chemical  Demilitarization-U.S.  Army 
Chemical  Materials  Agency.  According  to  the  SAR  for  each  program, 

O&S  costs  are  reported  in  other  sections  of  the  reports.  For  example, 
program  officials  told  us  that  O&S  costs  for  the  Assembled  Chemical 
Weapons  Alternatives  program  are  captured  in  research,  development, 
test,  and  evaluation  costs.  According  to  program  officials,  the  Chemical 
Demilitarization  program  is  a  one-of-a-kind  national  environmental  and 
safety  program  that  is  unlike  weapon  systems  that  report  SARs.  Further, 
officials  said  that  the  two  programs  have  not  separately  reported  any  O&S 
costs  since  they  were  designated  major  defense  acquisition  programs  in 
1994. 


SARs  are  intended  to  provide  Congress  with  authoritative  program 
information  on  the  cost,  schedule,  and  performance  of  major  weapon 
systems,  but  we  found  that  some  programs  submitted  unreliable  O&S 
cost  data.  More  specifically,  our  review  of  SAR  reports  for  the  15 
programs  in  our  sample  identified  inaccurate  cost  estimates  and  other 
errors  in  SARs  submitted  in  2007,  2009,  and  2010.  (As  noted  earlier, 
DOD  did  not  submit  SARs  in  2008.)  While  some  of  the  program  offices 
told  us  specific  reasons  for  the  errors,  others  did  not  provide  an 
explanation. 

Based  on  our  analysis  of  O&S  cost  data  reported  in  the  SAR  compared 
with  the  underlying  cost  estimates  and  other  information  provided  by  the 
program  offices,  we  found  that  7  of  the  15  programs  reported  inaccurate 
O&S  costs  in  one  or  more  of  the  three  annual  SARs. 


41  The  O&S  costs  of  the  Apache  Block  NIB  are  captured  in  the  SAR  for  the  Apache  Block 
MIA  Remanufacture  program.  According  to  program  officials,  the  programs  were  originally 
a  single  program  but  at  milestone  C  the  decision  was  made  to  split  them  into  two  separate 
programs.  The  Defense  Acquisition  Executive  approved  the  reporting  of  O&S  costs  in  the 
Apache  Block  IIIA  Remanufacture  SAR  only.  SARs  for  both  programs  note  that  the  O&S 
costs  are  reported  in  the  Apache  Block  IIIA  Remanufacture  SAR,  but  the  costs  are  not 
broken  out  between  the  two  programs. 


Program  Offices 
Sometimes  Provided 
Unreliable  Life-Cycle  O&S 
Cost  Estimates  in  Their 
SARs 
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•  The  F-35  Joint  Strike  Fighter  program  office  underreported  the 
average  cost  per  flying  hour  for  the  aircraft  in  the  2010  SAR.  The 
average,  steady-state  O&S  cost  per  flying  hour  was  reported  as 
$16,425  (fiscal  year  2002  dollars).  Program  officials  told  us  that  the 
number  of  aircraft  used  in  the  estimate  for  the  Air  Force’s  inventory 
was  not  accurate  and  the  estimate  also  did  not  project  for  future  cost 
growth  above  inflation.  The  estimate  included  approximately  528  extra 
aircraft  that  when  calculating  the  average  cost  per  flying  hour,  resulted 
in  higher  flight  hours  and  lower  average  costs  per  hour.  Further, 
according  to  the  SAR,  some  of  the  F-35’s  O&S  costs  were 
intentionally  excluded  from  the  estimate  to  enable  comparison  with 
the  antecedent  system,  the  F-16  C/D.  Costs  for  support  equipment 
replacement,  modifications,  and  indirect  costs  were  removed  from  the 
F-35’s  cost  per  flying  hour  since  they  were  not  available  for  the  F-1 6 
C/D.  Officials  calculated  that  the  revised  cost  per  flying  hour  for  the  F- 
35  was  $23,557  (fiscal  year  2002  dollars),  or  43  percent  higher,  after 
including  the  excluded  costs,  projecting  for  future  cost  growth  above 
inflation,  and  correcting  the  number  of  aircraft.  However,  they  noted 
that  the  total  O&S  life-cycle  cost  reported  in  the  SAR  for  the  F-35  was 
accurate  because  it  was  calculated  separately  from  the  average  cost 
per  flying  hour. 

•  The  Navy  Multiband  Terminal  program  office  underreported  estimated 
life-cycle  O&S  costs  in  the  2010  SAR.  The  program  reported  $219.1 
million  in  total  O&S  costs  but  excluded  an  additional  $591.3  million  for 
externally  funded  depot-level  repairables  ($148.4  million)  and  military 
personnel  ($442.9  million),  which  were  included  in  a  2010  service  cost 
estimate.42  Therefore,  only  27  percent  of  the  program’s  estimated 
total  O&S  costs  were  reported  in  the  2010  SAR.  Program  officials 
stated  that  these  costs  are  not  under  the  control  of  the  program  office 
and  should  not  be  reported  in  the  SARs.  However,  the  SAR  statute 
states  that  full  life-cycle  costs,  including  O&S  costs,  should  be 
reported  without  regard  to  funding  source  or  management  control. 

•  The  Air  Force  Joint  Primary  Aircraft  Training  System  program  office 
underreported  O&S  costs  in  the  2007  and  2009  SARs,  both  of  which 
were  based  on  a  2001  service  cost  estimate.  The  program,  which 
includes  the  T-6  aircraft  and  a  ground-based  training  system,  reported 


42  The  excluded  costs  in  the  service  cost  estimate  were  expressed  only  in  then-year 
dollars,  rather  than  constant  dollars.  Therefore,  all  cost  figures  in  this  example  are 
presented  in  then-year  dollars. 
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total  O&S  costs  of  $9.4  billion  (fiscal  year  2002  dollars)  in  both  SARs 
but  excluded  $2.1  billion  (fiscal  year  2002  dollars) — or  about  18 
percent — of  O&S  costs  for  the  program’s  ground-based  training 
system.  Program  officials  have  reported  the  same  O&S  costs  since 
the  annual  2002  SAR.  The  program,  which  updated  its  O&S  estimate 
in  201 1 ,  included  these  costs  in  the  total  O&S  costs  reported  in  the 
2010  SAR. 

•  The  Army’s  High  Mobility  Artillery  Rocket  System  program  office 
overstated  O&S  costs  in  the  program’s  2007,  2009,  and  2010  SARs. 
Although  program  office  estimates  were  provided  to  us  for  the  3 
years,  the  estimates  did  not  match  the  costs  reported  in  the  SARs. 

The  O&S  costs  reported  in  2007  were  higher  than  the  estimate  by 
$11.1  million  (fiscal  year  2003  dollars),  and  the  $988  million  (fiscal 
year  2003  dollars),  reported  in  both  2009  and  2010,  was  higher  than 
the  estimates  by  about  $300  million  (fiscal  year  2003  dollars),  or  about 
43  percent.  Program  officials  told  us  that  the  costs  had  been  reported 
incorrectly  in  each  year. 

•  The  JTRS  Handheld,  Manpack,  and  Small  Form  Fit  program 
underreported  total  O&S  costs  in  the  annual  2007  SAR.  The  SAR 
stated  that  the  O&S  costs  had  been  updated,  but  the  O&S  costs  were 
unchanged  from  prior  annual  SARs.  Program  officials  also  provided 
us  with  an  estimate  that  matched  the  numbers  reported  in  the  2007 
SAR.  When  asked  why  the  costs  had  not  changed,  program  officials 
stated  that  while  the  costs  for  procurement  and  research, 
development,  test,  and  evaluation  were  correctly  updated  in  2007,  the 
O&S  costs  were  not.  They  explained  that  the  reported  costs  of  $4.9 
billion  (fiscal  year  2004  dollars)  should  have  been  higher  by  $120 
million  (fiscal  year  2004  dollars),  but  they  did  not  provide  us  the 
estimate  on  which  that  higher  amount  was  based. 

•  The  Air  Force’s  Navstar  GPS  program,  as  noted  earlier,  did  not  report 
a  life-cycle  cost  estimate  in  the  annual  SARs  from  2007  through  2010. 
For  example,  according  to  the  2010  SAR,  the  O&S  costs  reported 
were  based  on  funding  for  fiscal  years  2008  through  2016.  Program 
officials  confirmed  that  the  O&S  amounts  reported  included  actual 
funding  for  the  current  year  and  funding  from  the  Air  Force’s  budget 
system  for  the  remaining  years.  However,  even  this  amount — about 
$469  million  (fiscal  year  2000  dollars)  in  2010,  for  example — was 
significantly  understated.  According  to  program  officials,  the  amount 
reported  in  the  SAR  is  only  60  percent  of  the  program’s  actual 
requirements  of  approximately  $782  million — a  difference  of  $31 3 
million — and  the  program  has  historically  been  funded  to  90  percent  of 
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requirements  with  supplemental  funds.  However,  this  was  not  noted  in 
the  SARs. 

•  The  FBCB2  program  underreported  total  O&S  costs  in  the  annual 
2007  through  2010  SARs.  As  explained  earlier,  reported  O&S  costs 
were  estimated  for  the  program’s  final  acquisition  milestone,  full-rate 
production,  in  2004.  In  subsequent  years,  however,  the  program’s 
procurement  quantities  increased  and  were  about  305  percent  higher 
in  the  2010  SAR  than  the  amount  used  to  develop  the  estimate. 
Further,  total  O&S  costs  of  $468  million  (fiscal  year  2005  dollars) 
reported  in  the  SARs  were  $129  million  less  than  the  $596.2  million 
estimated  in  2004.  Officials  initially  indicated  that  some  of  the 
estimated  O&S  costs  were  likely  included  with  the  program’s 
acquisition  costs  in  the  SAR,  but  they  were  unable  to  reconcile  the 
costs  in  the  two  documents. 

We  also  found  examples  of  inaccuracies  in  other  data  reported  in  the 
O&S  cost  section  of  the  SARs.  For  example,  the  2010  SAR  for  the  Joint 
MRAP  states  that  the  program’s  O&S  costs  were  reviewed  by  CAPE  in 
2010,  but  program  officials  and  prior-year  SARs  stated  that  the  review 
actually  occurred  in  2008.  Further,  neither  CAPE  nor  the  program  office 
was  able  to  provide  any  record  of  the  2008  review.  As  another  example, 
the  2010  SAR  for  the  F-22  indicates  that  the  reported  O&S  costs  were 
based  on  a  2004  acquisition  decision  estimate  that  was  updated  with 
analyses  from  2010  to  bring  the  estimate  in  line  with  the  current  approved 
F-22  production  program  and  operational  concepts.  However,  the  O&S 
costs  reported  are  identical  to  those  reported  in  the  2009  SAR,  which 
states  it  was  updated  based  on  analyses  from  2009. 

Implementation  of  the  GAO-identified  best  practices  already  discussed 
could  improve  the  reliability  of  O&S  costs  reported  in  the  SARs.  Together, 
the  best  practices  work  to  provide  more  assurance  that  the  correct 
information  is  reported.  For  example,  routinely  updating  O&S  cost 
estimates — and  related  SAR  data — will  likely  require  more  frequent 
changes  to  the  reported  cost  data.  Therefore,  it  is  less  probable  that  an 
error  or  omission  will  be  regularly  reported.  In  addition,  as  noted  earlier, 
comparing  a  program’s  cost  estimate  with  an  independent  cost  estimate, 
and  explaining  any  significant  differences,  could  help  decision  makers 
monitor  the  reasonableness  of  the  reported  data.  Finally,  tracking  O&S 
costs  over  time,  by  presenting  the  current  year  and  prior-year  program 
cost  estimates  and  explaining  significant  differences,  would  also  help  to 
test  the  reasonableness  of  reported  costs. 
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DOD  Could  Improve  O&S 
Cost  Reporting  with 
Detailed  Implementation 
Guidance  and  a  More 
Effective  SAR  Review 
Process 


DOD’s  reports  to  Congress  on  estimated  weapon  system  O&S  costs  were 
often  inconsistent  and  sometimes  unreliable  due  to  a  lack  of  detailed 
implementation  guidance  for  reporting  these  costs.  In  addition,  DOD’s 
process  for  reviewing  the  O&S  cost  sections  of  the  SAR  prior  to  final 
submission  did  not  provide  assurance  that  the  program  offices  reported 
costs  uniformly,  to  the  extent  practicable,  and  that  these  reported  costs 
were  reliable.  In  the  absence  of  improvements  to  the  SAR  guidance  and 
to  DOD’s  review  process,  deficiencies  in  reporting  estimated  life-cycle 
O&S  costs  are  likely  to  continue. 

DOD’s  existing  implementation  guidance  collectively  provides  minimal, 
and  in  some  areas  conflicting,  instructions  for  O&S  cost  reporting.  For 
example,  the  guidance  does  not  identify  which  cost  estimate  or  estimates 
should  be  used  to  report  O&S  costs  when  more  than  one  estimate  is 
available.  Often  multiple  cost  estimates  are  prepared  by  the  program 
office,  the  service,  and  CAPE  to  support  acquisition  decisions.  Further, 
DOD  officials  stated  that  O&S  costs  reported  in  the  SAR  should  be 
updated  only  at  acquisition  milestones.  Because  many  years  may  pass 
between  these  milestones,  however,  reported  O&S  costs  may  become 
outdated,  no  longer  reflecting  the  status  of  the  current  acquisition 
program. 

DOD’s  guidance  also  provides  very  little  detail  on  how  program  offices 
should  discuss  assumptions  underlying  the  cost  estimate.  DOD’s  draft 
SAR  policy,  for  example,  only  mentions  several  assumptions  for 
consideration,  such  as  operating  tempo,  expected  reliability  and 
maintainability  of  the  system,  the  maintenance  concept,  and  manning  and 
logistics  policies,  and  does  not  provide  specific  examples.  In  addition,  the 
statutory  SAR  requirement  to  report  all  program  costs,  regardless  of 
funding  source  or  management  control,  is  reflected  in  none  of  DOD’s 
SAR  implementation  guidance;  it  is  reflected  in  training  course  materials 
on  using  the  Defense  Acquisition  Management  Information  Retrieval 
system.  Finally,  DOD’s  draft  SAR  policy  provides  conflicting  instructions 
on  cost  reporting  for  antecedent  systems.  The  draft  policy  states  that 
antecedent  costs  should  be  reported  “whenever  those  costs  have 
previously  been  developed.”  However,  in  the  appendix,  the  draft  guidance 
states  that  O&S  costs  will  be  reported  for  antecedent  systems  “when  the 
replacement  system  is  required  to  report  O&S  costs.”  DOD  officials  could 
not  explain  the  reason  for  this  variance  in  the  guidance. 

While  some  program  offices  we  contacted  indicated  that  DOD’s 
implementation  guidance  on  reporting  O&S  costs  in  the  SAR  was 
sufficient,  officials  from  several  program  offices  in  our  sample  indicated 
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that  more  detailed  guidance  would  be  helpful  when  they  prepare  their 
annual  SAR  submissions.  These  officials  stated  that  there  was  minimal 
guidance  provided  on  what  should  be  included  in  the  O&S  narrative  and 
that  there  needed  to  be  more  consistency  in  SAR  O&S  reporting. 
Additionally,  they  explained  that  the  current  guidance  is  vague,  unclear, 
open  to  interpretation,  and  does  not  provide  useful  information  or 
examples  for  how  programs  should  be  reporting  these  costs.  Officials 
from  one  program  also  stated  that  there  is  no  direction  on  the  comparison 
of  program  costs  to  the  antecedent  system’s  costs,  so  the  approach  to 
making  this  comparison  is  open  to  interpretation.  They  noted  that  the 
guidance  does  not  specify  whether  the  program  office  should  alter  the 
weapon  system’s  O&S  costs  to  enable  a  true  comparison  with  the  costs 
for  the  antecedent  system,  or  whether  the  weapon  system’s  O&S  costs 
should  be  reported  without  modification.  Finally,  while  several  program 
offices  told  us  that  the  Defense  Acquisition  University  provides  useful 
training  on  acquisition  reporting  in  general,  they  said  that  the  materials 
should  be  more  readily  available  as  program  representatives  could  not 
always  attend  the  class  and  that  the  O&S  section  of  the  SAR  was  not 
covered  sufficiently. 

The  SAR  data  submitted  by  program  offices  are  subject  to  multiple 
reviews  within  the  military  services  and  by  OSD,  but  this  review  process 
has  not  provided  assurance  that  O&S  costs  are  reported  consistently  and 
reliably.  Although  our  review  did  not  include  a  full  evaluation  of  DOD’s 
SAR  review  process,  OSD  officials  explained  that  once  they  receive  the 
SAR  submissions,  there  is  a  relatively  short  amount  of  time  to  review  the 
SAR  O&S  data.  For  example,  according  to  the  SAR  review  schedule,  the 
Office  of  the  Assistant  Secretary  of  Defense  (Logistics  and  Materiel 
Readiness)  usually  has  about  a  week  to  review  the  O&S  cost 
submissions.  We  also  noted  that  “SAR  review  guidance”  that  is  included 
with  the  annual  memorandum  on  preparing  SARs  does  not  provide 
additional  direction  to  the  program  offices  on  what  to  include  in  their  O&S 
cost  submissions.  In  some  cases,  the  annual  memorandum  is  less 
specific  than  the  draft  SAR  policy.  The  deficiencies  in  DOD’s 
implementation  guidance  likely  hinder  the  effective  review  of  SAR  O&S 
cost  information  at  all  levels. 

The  department’s  emphasis  on  weapon  system  O&S  costs  has  been 
increasing  in  recent  years,43  but  the  primary  focus  continues  to  be  on 


43  This  increased  emphasis  is  discussed  in  GAO-10-717. 
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acquisition  costs.  According  to  OSD  acquisition  officials,  the  SAR  started 
as — and  is  still  often  viewed  as — primarily  an  acquisition  report.  This 
perspective  was  reflected  in  comments  from  some  program  officials.  For 
example,  officials  at  one  program  office  told  us  that,  due  to  a  constantly 
changing  acquisition  program,  their  time  was  largely  spent  on  estimating 
acquisition  costs.  Another  program  office  noted  that  the  focus  of  the  SAR 
statute  was  acquisition  costs  and  that  O&S  costs  will  vary  based  on 
emerging  needs.  Several  other  programs  indicated  that  O&S  cost 
estimating  was  not  particularly  useful,  as  their  systems  had  not  yet 
entered  into  production  or  sustainment,  and  actual  cost  data  were  either 
not  yet  available  or  could  not  be  obtained  by  the  program  office.  Finally, 
other  program  offices  stated  that  since  they  do  not  fund  the  support  of  the 
weapon  system,  the  O&S  cost  estimates  should  be  done  by  the 
organizations  responsible  for  providing  this  funding. 


Conclusions 


Without  more  consistent  and  reliable  reporting  of  estimated  weapon 
system  O&S  costs,  Congress  and  senior  DOD  officials  may  have  limited 
visibility  of  information  needed  to  effectively  oversee  the  full  life-cycle 
costs  associated  with  weapon  system  acquisitions.  Improvements  in  the 
reporting  of  these  data  could  provide  a  more  complete  picture  of  the 
potential  total  financial  commitment  being  made  to  these  systems  over  a 
period  lasting  many  decades.  SAR  cost  estimates  are  reported  early 
during  acquisition,  when  there  is  the  greatest  chance  for  managing  or 
reducing  future  O&S  costs.  By  facilitating  inquiries  on  changes  from  prior 
cost  estimates  and  cost  drivers,  such  information  could  affect  acquisition 
investment  decisions  and  result  in  tradeoffs  that  otherwise  might  not  be 
considered.  Furthermore,  improvements  to  SAR  reporting  would  be 
consistent  with  a  provision  in  the  National  Defense  Authorization  Act  for 
Fiscal  Year  2012  directing  DOD  to  take  actions  aimed  at  better 
assessing,  managing,  and  controlling  weapon  system  O&S  costs. 


Recommendations  for  'mProve  visibility  over  estimated  life-cycle  O&S  costs  during  weapon 

system  acquisition,  we  recommend  that  the  Secretary  of  Defense  take  the 
Executive  Action  following  two  actions. 

First,  we  recommend  that  the  Secretary  of  Defense  direct  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  to  revise 
DOD’s  guidance  for  implementing  statutory  SAR  requirements.  The 
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revisions,  at  a  minimum,  should  provide  additional  detail  on  the  following 
areas: 

•  the  explanatory  information  that  should  be  included  in  the  O&S 
narrative,  including  the  specific  assumptions  underlying  the  cost 
estimate; 

•  the  source  to  be  used  as  the  basis  for  reported  O&S  cost  estimate 
information,  especially  when  more  than  one  source  is  available  (such 
as  a  program  office  cost  estimate,  service  cost  estimate,  and  CAPE 
independent  cost  estimate); 

•  a  consistent  unit  of  measure  for  reporting  average  costs  over  time  by 
commodity  type — or  other  designated  weapon  system  group — as 
agreed  to  by  OSD  and  the  services; 

•  criteria  for  identifying  an  antecedent  system  and  reporting  on  the 
results  of  the  cost  comparison  in  the  SAR;  and 

•  reporting  O&S  costs  for  major  modifications  to  existing  weapon 
systems. 

In  revising  the  guidance,  the  Under  Secretary  of  Defense  should 
incorporate  best  practices  for  preparing  and  presenting  cost  estimates, 
including: 

•  a  comparison  of  current-year  to  prior-year  O&S  cost  estimates;  the 
identification  of  cost  drivers  that  resulted  in  changes  in  these 
estimates,  if  significant;  and  the  level  of  detail  that  should  be  reported; 

•  a  comparison  of  the  reported  cost  estimate  with  the  most  recent 
independent  cost  estimate,  along  with  an  explanation  of  any 
significant  differences  between  the  two  estimates;  and 

•  The  frequency  with  which  O&S  costs  reported  in  the  SAR  should  be 
updated,  including  guidance  on  what  changes  in  the  program’s  status 
should  trigger  an  update. 

Second,  we  also  recommend  that  the  Secretary  of  Defense  direct  that  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics,  in 
conjunction  with  the  Secretaries  of  the  Army,  the  Air  Force,  and  the  Navy, 
evaluate  the  current  review  process,  identify  any  weaknesses,  and 
institute  corrective  actions  as  needed  to  provide  greater  assurance  that 
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estimated  life-cycle  O&S  costs  included  in  the  SAR  reports  submitted  by 
program  offices  consistently  follow  the  implementation  guidance, 
including  any  revisions  to  the  guidance  as  described  above,  and  report 
reliable  cost  data.  As  part  of  this  evaluation,  DOD  should  consider 
whether  additional  steps  are  necessary  for  the  department  to  enhance  the 
emphasis  placed  on  reporting  estimated  life-cycle  O&S  costs  in  the  SAR. 


DOD  provided  comments  on  a  draft  of  this  report.  In  its  comments,  DOD 
agreed  with  both  of  our  recommendations.  The  department’s  written 
comments  are  reprinted  in  appendix  III.  DOD  also  provided  technical 
comments  that  we  have  incorporated  into  this  report  where  appropriate. 

In  concurring  with  our  first  recommendation  to  revise  DOD’s  guidance  for 
implementing  statutory  SAR  requirements,  DOD  noted  that  the  focus  of 
the  SAR  has  always  been  primarily  on  acquisition  rather  than 
sustainment.  DOD  further  stated  that  Congress,  in  requiring  DOD  to  add 
O&S  costs  to  the  SAR  report,  did  not  intend  for  DOD  to  develop  O&S 
costs  for  each  submission  but  to  report  the  latest  available  estimate  for 
the  program.  Our  report  recognizes  that  the  development  of  new  O&S 
cost  estimates  is  not  required  for  each  annual  SAR  submission.  However, 
these  costs  represent  a  significant  proportion  of  a  system’s  total  costs 
over  its  life  cycle.  Moreover,  we  found  that  the  timing  of  updates  to  the 
O&S  costs  reported  in  the  SAR  varied  widely,  as  DOD  has  not  identified 
what  changes  in  a  program's  status — other  than  established  acquisition 
milestones,  which  can  be  many  years  apart — should  trigger  such 
updates.  We  also  continue  to  believe  that  DOD  needs  to  clearly  identify 
the  source  and  date  of  the  O&S  cost  estimate  data  reported  in  the  SAR. 
Our  recommendations  reflect  these  and  other  weaknesses  in  the  current 
reporting  of  O&S  costs. 

DOD’s  comments  identified  actions  it  plans  to  take  to  implement  our 
recommendations.  DOD  stated  that  it  will  expand  and  update  its  current 
guidance  for  the  O&S  cost  section  of  the  SAR,  as  contained  in  the 
Defense  Acquisition  Guidebook.  DOD  plans  to  make  revisions  specifically 
with  regard  to  assumptions  and  ground  rules  (e.g.,  the  source  and  date  of 
the  estimate  reported);  a  consistent  unit  of  measure  for  reporting  O&S 
costs  for  each  type  of  commodity;  identifying,  and  reporting  on, 
antecedent  systems;  and  reporting  O&S  costs  for  major  modifications. 
These  planned  revisions  to  the  guidance  are  positive  steps.  We  plan  to 
monitor  DOD’s  actions  as  part  of  our  recommendation  follow-up  process. 


Agency  Comments 
and  Our  Evaluation 
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Regarding  other  revisions  to  the  guidance  that  we  recommend  to 
incorporate  best  practices  for  O&S  cost  reporting,  DOD  stated  that  the 
department  is  not  yet  in  a  position  to  add  a  credible  O&S  cost  variance 
analysis.  Although  DOD  does  not  define  what  it  means  by  "cost  variance 
analysis,"  it  is  reasonable  to  expect  that  such  analysis  would  involve 
comparing  changes  from  a  previous  cost  estimate  and  identifying  any 
significant  cost  drivers.  DOD  noted  that  it  is  implementing  new  O&S- 
related  requirements  from  the  National  Defense  Authorization  Act  for 
Fiscal  Year  2012,  as  well  as  previous  requirements  from  the  Weapon 
Systems  Acquisition  Reform  Act  of  2009,  including  requirements  that  deal 
with  cost  variance  analysis.  DOD  stated  that  it  is  premature  to  determine 
to  what  extent  DOD's  implementation  of  these  requirements  will  affect  the 
reporting  of  O&S  costs  in  the  SAR.  With  these  and  other  ongoing 
activities  related  to  the  management  and  control  of  O&S  costs,  DOD 
would  prefer  to  defer  these  additional  reporting  requirements  for  the  SAR 
for  now. 

We  are  aware  that  DOD  has  a  number  of  ongoing  activities  to  improve 
the  management  and  control  of  O&S  costs  and  must  respond  to  several 
new  requirements,  as  stated  in  DOD's  comments.  For  example,  the  O&S- 
related  guidance  required  by  the  National  Defense  Authorization  Act  for 
Fiscal  Year  2012  must  be  issued  within  180  days  from  the  date  the  Act 
was  enacted,  which  was  December  31 ,  201 1 .  If  such  activities  result  in 
improved  visibility  of  O&S  costs  within  the  department,  and  DOD 
coordinates  these  activities  with  efforts  to  improve  O&S  cost  reporting  in 
the  SAR,  then  we  agree  that  it  may  be  preferable  to  delay  implementation 
of  the  best  practices  we  recommend  in  our  report.  However,  we  continue 
to  believe  that  these  best  practices,  when  implemented,  will  provide  better 
information  on  the  current  status  and  direction  of  long-term  O&S  costs 
and  thus  can  improve  congressional  oversight  of  weapon  system  costs. 
Therefore,  these  elements  of  our  recommendation  remain  valid. 

DOD  also  concurred  with  our  second  recommendation  to  evaluate  and 
make  any  changes  needed  to  strengthen  its  current  process  for  reviewing 
O&S  cost  reporting  prior  to  submission  of  SARs  to  Congress.  In  its 
comments,  DOD  cited  actions  it  would  take  in  the  short  term  to  improve 
the  review  of  O&S  costs  prior  to  submission  of  SAR  reports  at  the  end  of 
March  2012.  DOD  stated  that  the  O&S  cost  section  will  be  given 
additional  emphasis  during  this  reporting  period.  Subsequently,  DOD  will 
convene  a  joint  OSD/DOD  component  working  group  that  will  evaluate 
the  current  SAR  review  process,  identify  any  weaknesses,  and  institute 
corrective  actions  as  needed  to  improve  the  data  quality  for  the  estimated 
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life-cycle  O&S  costs  reported  in  the  SAR.  We  believe  these  actions,  when 
implemented,  will  meet  the  intent  of  our  recommendation. 


We  are  sending  copies  of  this  report  to  interested  congressional 
committees;  the  Secretary  of  Defense;  the  Secretaries  of  the  Army,  the 
Navy,  and  the  Air  Force;  the  Acting  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics;  and  the  Director,  Office  of 
Management  and  Budget.  In  addition,  the  report  will  be  available  at  no 
charge  on  the  GAO  website  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (404)  679-1808  or  russellc@gao.gov.  Contact  points  for  our 
Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on 
the  last  page  of  this  report.  Key  contributors  are  listed  in  appendix  IV. 


Acting  Director,  Defense  Capabilities  and  Management 
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Appendix  I:  Scope  and  Methodology 


To  determine  the  extent  to  which  the  selected  acquisition  reports  (SAR) 
provide  consistent  and  reliable  operating  and  support  (O&S)  cost 
estimate  information  that  enables  effective  oversight  of  major  weapon 
system  costs,  we  reviewed  statutory  requirements  in  10  U.S.C.  §  2432  for 
reporting  weapon  system  life-cycle  costs  in  the  SARs,  as  well  as 
Department  of  Defense  (DOD)  implementation  guidance  for  the  SAR.  We 
also  reviewed  DOD  guidance  for  preparing  weapon  system  O&S  costs 
and  GAO-identified  cost-estimating  best  practices  to  identify  the  scope 
and  nature  of  cost  estimate  information  needed  for  effective  program 
management  and  oversight. 

We  interviewed  and  obtained  documentation  from  DOD  and  military 
service  officials  responsible  for  weapon  system  acquisition,  logistics,  and 
cost  analysis  to  understand  DOD’s  approach  and  process  for  reporting 
O&S  cost  estimates  in  the  SARs.  Offices  we  contacted  included  the 
following: 

•  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics 

•  Office  of  the  Director,  Acquisition  Resources  and  Analysis 

•  Office  of  the  Deputy  Assistant  Secretary  of  Defense  for  Materiel 
Readiness 

•  Office  of  the  Director,  Cost  Assessment  and  Program  Evaluation 

•  Office  of  the  Deputy  Assistant  Secretary  of  the  Army,  Cost  and 
Economics 

•  Naval  Center  for  Cost  Analysis 

•  Air  Force  Cost  Analysis  Agency 

We  obtained  SARs  for  all  95  weapon  systems  that  reported  a  December 
2010  SAR.1  These  reports  were  contained  in  the  Defense  Acquisition 
Management  Information  Retrieval  system,  which  is  a  web-based  system 
used  within  DOD  to  collect  and  maintain  SAR  information  submitted  by 
program  offices.  We  determined  that  the  data  in  this  system  accurately 
reflected  information  submitted  by  weapon  system  program  offices  and 
therefore  were  sufficiently  reliable  for  the  purposes  of  our  analysis.  After 
determining  that  a  total  of  84  of  the  95  weapon  systems  included  O&S 
costs  in  their  December  2010  SARs,  we  analyzed  the  annual  SARs  that 


1  Although  a  total  of  96  programs  submitted  a  SAR  in  2010,  the  Ballistic  Missile  Defense 
Program  had  not  yet  reached  milestone  B  and  thus  was  not  required  to  report  O&S  costs. 
For  this  reason,  we  did  not  include  this  program  in  our  analyses. 


Page  33 


GAO-12-340  Defense  Logistics 


Appendix  I:  Scope  and  Methodology 


were  submitted  for  these  systems  between  2005  through  2010. 2 
Specifically,  we  analyzed  the  SARs  to  determine  the  types  and  scope  of 
explanatory  information  included  in  the  O&S  narrative  accompanying  the 
cost  estimate  data;  the  source  of  the  O&S  cost  estimate  cited  as  the  basis 
for  the  reported  costs;  the  units  of  measure  used  to  present  O&S  costs; 
the  frequency  that  O&S  costs  were  updated  from  year  to  year;  and  the 
extent  to  which  O&S  costs  for  antecedent  systems  were  reported.  We 
compared  the  SARs  across  each  of  these  categories  to  determine  the 
extent  to  which  information  was  reported  consistently  across  all  84 
weapon  systems. 

From  the  population  of  84  weapon  systems  that  included  O&S  cost 
estimates  in  the  2010  SARs,  we  selected  a  sample  of  15  weapon 
systems  for  further  analysis.3  We  designed  the  sample  to  ensure  that  a 
range  of  weapon  systems  were  represented  based  on  commodity  type 
and  service  responsible  for  managing  the  program.  We  selected  three  or 
four  weapon  systems  per  service  and  at  least  one  commodity  type  within 
each  service  for  a  total  sample  size  of  15.4  We  also  examined  the 
distribution  of  weapon  systems’  total  costs  across  our  sample  selection  in 
terms  of  both  dollars  and  the  upper  and  lower  50  percent  of  weapon 
systems  that  reported  O&S  costs  in  the  2010  SAR.  We  determined  that 
the  sample  contained  an  adequate  mix  of  high-  and  low-dollar  weapon 
systems  for  our  purposes.  The  results  from  this  nonprobability  sample 
cannot  be  used  to  make  inferences  about  all  major  weapon  systems 
because  the  sample  may  not  reflect  all  characteristics  of  the  population. 
The  15  programs  in  our  sample  are  shown  in  table  3. 


2  Annual  SARs  are  submitted  to  Congress  within  60  days  after  the  date  on  which  the 
President’s  Budget  is  submitted  to  Congress  for  the  following  fiscal  year.  We  analyzed  the 
December  SARs  for  the  years  2005,  2006,  2007,  2009,  and  2010.  Annual  SARs  were  not 
submitted  in  2008. 

3  Our  original  sample  included  16  weapon  systems.  However,  one  system,  the  Army’s 
Increment  1  Early-lnfantry  Brigade  Combat  Team  program,  was  subsequently  canceled. 

4  We  categorized  systems  as  aircraft,  ship,  ground,  or  “other”  (e.g.,  missile  programs  and 
command  and  control  systems). 
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Table  3: 

Major  Weapon  Systems  in  GAO’s  Nonprobability  Sample 

Service 

Major  weapon  system 

Commodity  type 

Army 

1.  Force  XXI  Battle  Command  Brigade  and  Below 

Other 

2.  High  Mobility  Artillery  Rocket  System 

Other 

3.  MQ-1C  Unmanned  Aircraft  System  Gray  Eagle 

Aircraft 

Navy 

4.  Joint  Mine  Resistant  Ambush  Protected 

Land 

5.  LHA  6  America  Class 

Ship 

6.  Navy  Multiband  Terminal 

Other 

7.  V-22  Joint  Services  Advanced  Vertical  Lift  Aircraft  (Osprey) 

Aircraft 

Air  Force  8.  F-22  Raptor 

Aircraft 

9.  Joint  Primary  Aircraft  Training  System 

Aircraft 

10.  Navstar  Global  Positioning  System 

Other 

11.  National  Polar-orbiting  Operational  Environmental  Satellite  System 

Other 

DOD 

12.  Airborne  and  Maritime/Fixed  Station  Joint  Tactical  Radio  System 

Other 

13.  F-35  Joint  Strike  Fighter 

Aircraft 

14.  Joint  Tactical  Radio  System  Ground  Mobile  Radios 

Other 

15.  Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form  Fit 

Other 

Source:  GAO  analysis  of  2010  SAR  data. 


Through  a  data  collection  instrument,  interviews,  and  other  contacts,  we 
obtained  information  from  the  15  program  offices  on  the  O&S  costs  they 
had  reported  in  the  SAR,  including  the  cost  estimates,  where  available, 
that  formed  the  basis  for  their  reported  costs.  We  compared  the  cost 
estimates  used  to  develop  SAR  O&S  costs,  as  well  as  any  additional  cost 
estimates  that  had  been  developed  for  the  weapon  systems,  to  the  O&S 
costs  reported  in  the  SARs.  We  interviewed  program  officials  about  the 
reasons  for  any  discrepancies  we  identified  between  the  SAR  O&S  costs 
and  the  cost  estimates.  For  those  weapon  systems  that  did  not  include 
O&S  cost  estimates  in  the  2010  SARs,  we  contacted  the  individual 
program  offices  to  obtain  information  about  the  reasons  why  these  costs 
were  not  included.  These  programs  are  listed  in  table  2  of  the  report. 

We  conducted  this  performance  audit  from  February  201 1  to  February 
2012  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 
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Appendix  II:  DOD’s  Guidance  Implementing 
the  Statutory  SAR  Requirements 


This  appendix  provides  additional  information  on  DOD’s  guidance  that 
implements  the  statutory  SAR  requirements  in  10  U.S.C.  §  2432. 

DOD  has  issued  various  guidance  documents  that  implement  the 
statutory  SAR  requirements.  DOD  Instruction  5000.02, 1  which  addresses 
the  operation  of  the  defense  acquisition  system,  includes  guidance  on 
SARs  that  is  similar  to  the  basic  statutory  requirements  in  10  U.S.C.  § 
2432.  The  guidance,  for  example,  states  that  SARs  should  be  submitted 
at  program  initiation  (normally  milestone  B  except  for  some  ship 
programs)  or  at  the  time  that  the  program  is  designated  as  a  major 
defense  acquisition  program.  It  reiterates  that  programs  shall  report 
annually,  with  the  exception  of  quarterly  reports  that  are  required  when 
acquisition  costs  increase  or  schedules  slip.  Further,  the  instruction 
requires  the  submission  of  quarterly  SARs  after  the  program  rebaselining 
that  occurs  after  a  major  milestone  decision  (i.e. ,  milestone  C  or 
milestones  B  and  C  for  some  ship  programs). 

Another  source  of  guidance  on  SAR  reporting  is  the  Defense  Acquisition 
Guidebook,2  which  describes  discretionary  best  practices  for  acquisition 
professionals  to  consider  while  meeting  various  reporting  requirements 
throughout  the  acquisition  process.  The  guidebook  contains  a  section 
summarizing  the  statutory  requirements  for  SAR  content  and  submission 
and  reiterates  that  a  full  life-cycle  analysis  of  costs  should  be  reported  for 
programs,  including  each  evolutionary  increment,  as  available,  and  for 
antecedent  programs,  if  applicable. 

DOD’s  SAR  policy,3  issued  in  draft  in  2006  and  never  finalized,  further 
identifies  information  that  should  be  included  in  the  O&S  cost  section  of 
the  SAR.  According  to  the  draft  policy,  programs  that  have  reached 
milestone  B  should  report  the  most  recent  estimate  of  O&S  costs,  such  as 
those  developed  for  service  cost  reviews  and  acquisition  milestone 
decisions,  and  should  include  both  the  source  and  date  of  this  estimate  in 
the  SAR.  Average  annual  O&S  costs  should  be  reported  in  constant, 
base-year  dollars  in  a  unit  of  measure  (for  example,  cost  per  aircraft, 
squadron,  or  wing)  determined  by  the  service.  The  draft  policy  further 


1  DOD  Instruction  5000.02,  Operation  of  the  Defense  Acquisition  System  (Dec.  8,  2008). 

2  DOD,  Defense  Acquisition  Guidebook  (201 1 ). 

3  OSD,  Selected  Acquisition  Report  (draft)  (2006).  Although  issued  in  draft,  the  policy  was 
never  issued  in  final  form 
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states  that  assumptions  underlying  the  estimate  should  be  included. 
Operating  tempo,  expected  reliability  and  maintainability  of  the  system, 
maintenance  concept,  and  manning  and  logistics  policies  are  provided  as 
examples  of  the  estimate  assumptions  that  should  be  included  in  the 
SAR.  Finally,  the  draft  policy  states  that  programs  should  report  the  total 
estimated  O&S  costs,  and  estimate  assumptions,  for  an  antecedent 
system  if  one  has  been  identified  and  these  costs  were  previously 
developed  for  that  system. 

Each  year  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics  issues  a  memorandum  to  the  military  services  that  provides 
guidance  for  preparing  the  annual  SARs,  including  instructions  for 
programs  that  have  reached  milestone  B  and  are  required  to  report  O&S 
costs.4  For  fiscal  years  2007,  2009,  and  2010,  this  annual  guidance 
states  that  programs  should  report  total  estimated  O&S  costs  in  both 
constant  and  then-year  dollars,  and  that  the  assumptions  that  formed  the 
basis  of  the  estimate  and  the  date  of  the  estimate  should  be  included. 
Further,  programs  should  report  an  average  unit  of  measure  (e.g., 
average  annual  cost  per  squadron,  average  annual  cost  per  system)  for 
the  O&S  costs  of  both  the  current  weapon  system  and  the  antecedent 
system  in  constant  dollars.  If  there  is  no  antecedent  system,  this  should 
be  stated  in  the  narrative  of  the  O&S  cost  section.  If  there  is  an 
antecedent  system  but  the  data  are  not  currently  available,  the 
antecedent  system  should  be  identified  in  the  narrative,  along  with  a 
statement  that  the  required  data  are  not  available  (e.g.,  “the  O&S  costs 
for  the  antecedent  system  are  not  currently  available,  but  will  be  provided 
in  the  next  SAR”).  Finally,  programs  should  explain  in  the  narrative  how 
the  average  annual  costs  were  calculated  using  the  estimated  O&S  cost 
total. 

Finally,  DOD  has  developed  instructions5  and  training  for  entering  SAR 
data  into  the  Defense  Acquisition  Management  Information  Retrieval 
system.  DOD  instructions  for  entering  SAR  data  into  the  system  generally 
align  with  the  2006  draft  policy.  The  Defense  Acquisition  University  offers 


4  At  the  time  we  conducted  our  review,  the  most  recent  annual  guidance  was  issued  in 
January  2011.  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics, 
Memorandum  for  Assistant  Secretaries  of  the  Military  Services,  December  2010  Selected 
Acquisition  Reports  (SARs)  Guidance  (Jan.  14,  201 1 ). 

5  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics,  SAR  Data  Entry 
Instructions  (draft)  (Nov.  5,  2010). 
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SAR  training  on  using  this  system.  According  to  Office  of  the  Secretary  of 
Defense  (OSD)  officials,  the  primary  class,  Acquisition  Reporting  for 
Major  Defense  Acquisition  Programs  and  Major  Automated  Information 
Systems,  is  usually  offered  in  January  and  October.  During  the  4-day 
class,  participants  receive  step-by-step  instruction  on  report  preparation 
using  the  system’s  web  application.  The  training  materials  include  basic 
SAR  O&S  cost  reporting  information.  For  example,  estimate  assumptions 
should  be  reported,  calculation  of  average  costs  from  total  O&S  costs 
should  be  provided,  and  costs  should  always  be  updated  at  major 
acquisition  milestones.  The  training  materials  reiterate  that  costs  should 
include  both  direct  and  indirect  costs,  regardless  of  funding  source  or 
management  control. 


Page  38 


GAO-12-340  Defense  Logistics 


Appendix  III:  Comments  from  the 
Department  of  Defense 


ASSISTANT  SECRETARY  OF  DEFENSE 
3500  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3500 


JAN  2  7  2012 


Mr.  Cary  B.  Russell 

Director,  Defense  Capabilities  and  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Mr.  Russell: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  draft  report 
GAO-12-340,  “DEFENSE  LOGISTICS:  Improvements  Needed  to  Enhance 
Oversight  of  Estimated  Long-Term  Costs  for  Operating  and  Supporting  Major 
Weapon  Systems,”  dated  December  21,  201 1  (GAO  Code  351582).  Detailed 
comments  on  the  report  recommendations  are  enclosed. 


Sincerely, 


Alan  RrEsteTez 


Enclosure: 
As  stated 
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GAO  DRAFT  REPORT  DATED  DECEMBER  21,  2011 
GAO-12-340  (GAO  CODE  351582) 

DEFENSE  LOGISTICS:  IMPROVEMENTS  NEEDED  TO  ENHANCE  OVERSIGHT 
OF  ESTIMATED  LONG-TERM  COSTS  FOR  OPERATING  AND  SUPPORTING 
MAJOR  WEAPON  SYSTEMS” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 


RECOMMENDATION  I:  To  improve  visibility  over  estimated  life-cycle  O&S  costs  during 
weapon  system  acquisition,  GAO  recommends  that  the  Secretary  of  Defense  direct  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  to  revise  DoD's  guidance  for 
implementing  statutory  SAR  requirements.  The  revisions,  at  minimum,  should  provide  additional 
detail  regarding  the  following  areas: 

•  The  explanatory  information  that  should  be  included  in  the  O&S  narrative,  including  the 
specific  assumptions  underlying  the  cost  estimate. 

•  The  source  to  be  used  as  the  basis  for  reported  O&S  cost  estimate  information,  especially 
when  more  than  one  source  is  available  (such  as  a  program  office  cost  estimate,  service 
cost  estimate,  and  CAPE  independent  cost  estimate). 

•  A  consistent  unit  of  measure  for  reporting  average  costs  over  time  by  commodity  type  - 
or  other  designated  weapon  system  group  -  as  agreed  to  by  OSD  and  the  services. 

•  Criteria  for  identifying  an  antecedent  system  and  reporting  on  the  results  of  the  cost 
comparison  in  the  SAR. 

•  Reporting  O&S  costs  for  major  modifications  to  existing  weapon  systems. 

In  revising  the  guidance,  the  Under  Secretary  of  Defense  should  incorporate  best  practices  for 
preparing  and  presenting  cost  estimates,  including: 

•  A  comparison  of  current  year  to  prior  year  O&S  cost  estimates:  the  identification  of  cost 
drivers  that  resulted  in  changes  in  these  estimates,  if  significant;  and  the  level  of  detail 
that  should  be  reported. 

•  A  comparison  of  the  reported  cost  estimate  with  the  most  recent  independent  cost 
estimate,  along  with  an  explanation  of  any  significant  differences  between  the  two 
estimates. 

•  The  frequency  with  which  O&S  costs  reported  in  the  SAR  should  be  updated,  including 
guidance  on  what  changes  in  the  program's  status  should  trigger  an  update. 

DoD  RESPONSE:  Concur  with  comment.  Since  1969,  when  the  Department  first  began 
submitting  Selected  Acquisition  Reports  to  Congress,  the  report  has  always  been  primarily  an 
acquisition,  not  a  sustainment,  report.  When  Congress  added  O&S  costs  to  the  SAR  in  1985,  it 
was  made  clear  in  the  legislative  history  that  the  Department  should  not  develop  O&S  cost 
estimates  for  each  submission  but  report  the  latest  available  estimate  for  the  program.  Unlike  the 
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acquisition  portion  of  the  SAR  that  is  updated  with  each  submission  to  reflect  actual  costs 
expended,  funds  budgeted,  and  current  projections  of  costs  to  go,  the  O&S  section  of  the  SAR 
reports  only  estimates  of  O&S  costs.  Therefore,  since  inception,  the  O&S  costs  in  the  SAR 
represent  the  latest  available  estimate  from  the  DoD  cost  community. 

The  Department  concurs  with  the  GAO  recommendation  to  expand  and  update  the  current 
guidance  for  the  O&S  cost  section  of  the  SAR,  which  is  contained  in  the  Defense  Acquisition 
Guidebook  (DAG).  DoD  will  revise  this  guidance  for  the  DAG  to  include  the  following: 

-  More  specific  direction  as  to  what  to  include  in  Assumptions  and  Ground  Rules  (e.g., 
source  and  date  of  estimate  reported). 

-  A  consistent  unit  of  measure  for  unitizing  O&S  costs  for  each  commodity.  [The 
December  201 1  SAR  guidance  has  already  directed  that  DoD  Components  transition  to 
uniform  units  of  measure  for  each  commodity.] 

-  Additional  criteria  for  identifying  and  reporting  antecedent  systems.  [DoD  will  retain  the 
longstanding  definition  of  an  antecedent  being  one  that  is  replaced  by  the  weapon 
system.] 

-  Reporting  O&S  costs  for  major  modifications.  [The  guidance  will  reflect  our  current 
practice  of  only  reporting  significant  differential  O&S  costs  for  modification  programs] 

However,  the  Department  is  not  in  a  position  to  add  a  credible  O&S  cost  variance  analysis  to  the 
SAR  at  this  time.  DoD  is  currently  implementing  detailed  O&S  -  related  legislation  contained  in 
Section  832  of  the  FY  2012  National  Defense  Authorization  Act  (NDAA)  and  previously  from 
the  Weapon  Systems  Acquisition  Reform  Act  of  2009.  A  portion  of  this  legislation  deals  with 
recording  the  reasons  for  O&S  cost  variances,  as  well  as  the  requirement  for  establishing  periodic 
updates  of  O&S  cost  estimates  throughout  the  life  cycle  of  a  weapon  system.  It  is  premature  to 
determine  to  what  extent  DoD’s  implementation  of  this  new  legislation  will  involve  the  pre¬ 
existing  requirement  for  reporting  O&S  costs  in  the  SAR.  There  is  a  wide  range  of  ongoing 
activities  related  to  the  management  and  control  of  O&S  costs  that  exist  within  or  internal  to  the 
Department.  In  order  to  reduce  any  duplicative  efforts,  we  would  prefer  to  defer  these  additional 
reporting  requirements  for  the  SAR  for  now. 


RECOMMENDATION  2:  GAO  also  recommends  that  the  Secretary  of  Defense  direct  that  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  in  conjunction  with  the 
Secretaries  of  the  Army,  the  Air  Force,  and  the  Navy,  evaluate  the  current  review  process, 
identify  any  weaknesses,  and  institute  corrective  actions  as  needed  to  provide  greater  assurance 
that  estimated  life-cycle  O&S  costs  included  in  the  SAR  reports  submitted  by  program  offices 
consistently  follow  the  implementation  guidance,  including  any  revisions  to  the  guidance  as 
described  above,  and  report  reliable  cost  data.  As  part  of  this  evaluation,  DoD  should  consider 
whether  additional  steps  are  necessary  for  the  department  to  enhance  the  emphasis  placed  on 
reporting  estimated  life-cycle  O&S  costs  in  the  SAR. 

DoD  RESPONSE:  Concur.  The  Department  is  continually  seeking  to  improve  the  review 
processes  for  the  SAR.  For  the  December  201 1  SAR  reporting  period,  we  plan  to  have  an  O&S 
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subject  matter  expert  from  the  Office  of  the  Assistant  Secretary  of  Defense  for  Logistics  and 
Materiel  Readiness  attend  each  of  the  joint  Office  of  the  Secretary  of  Defense  (OSD)/DoD 
Component  program  reviews.  These  detailed  reviews  will  be  conducted  in  early  March  2012  to 
incorporate  all  OSD  comments  into  the  SARs  prior  to  their  submission  to  Congress  on  March  29. 
2012.  The  O&S  cost  section  will  be  given  additional  emphasis  during  this  reporting  period. 

Subsequent  to  the  issuance  of  updated  SAR  guidance  and  the  lessons  learned  from  the  increased 
focus  on  O&S  costs  reported  in  the  December  201 1  SARs,  the  Department  will  convene  a  Joint 
OSD/DoD  Component  working  group  that  will  evaluate  the  current  SAR  review  process,  identify 
any  weaknesses,  and  institute  corrective  actions,  as  needed,  to  improve  data  quality  for  the 
estimated  life  cycle  O&S  costs  reported  in  the  SAR. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (www.gao.gov).  Each  weekday  afternoon, 
GAO  posts  on  its  website  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  “E-mail  Updates.” 
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white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 

http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 
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Visit  GAO  on  the  web  at  www.gao.gov. 
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